3. Memory Management

3.1. Changing the Memory Allocator [Recommended]

The NEX library frequently performs dynamic memory allocation. Use the MemoryManager::SetBasicMemoryFunctions function to set a memory allocator prepared in the application. If no memory allocator is set, the NEX library will allocate memory by using the malloc and free functions, but this is for debugging. It is recommended that an application-specific memory management mechanism be used in the retail product.

If a memory allocator has been set, it is used by the NEX library for dynamic memory allocation, and by the application when new is done on public classes that inherit RootObject. (If a class does not inherit from RootObject, that fact is noted in its description.)

3.2. Handling Out of Memory Conditions [Required]

The NEX library includes code that presumes dynamic memory allocation always succeeds. For this reason, make sure that your implementation returns a valid memory address. Specifically, it needs to be implemented as follows.

  • Shut down the library if you allocate more memory than expected.
  • Allocate a memory parachute of several KB beforehand, enough to run until the library can be shut down normally. If an allocation fails, deallocate the memory parachute. Then shut down the library.
  • Have plenty of memory available beforehand. (Perform enough debugging to make sure that there is enough memory if the communication environment worsens, or in other scenarios.)

3.3. Packet Buffer Manager Settings (Recommended)

One source of dynamic memory allocation in the NEX library is the packet buffer manager, which manages packet buffer memory in P2P communication. Configure the packet buffer manager to suit the way that your application sends and receives data. For more information, see section 3.8 Managing Packet Buffer Memory.

3.4. Thread Safety Considerations

If you use the NEX API from multiple threads, you must use a thread-safe allocator because memory is also allocated from multiple threads. For more information, see section 4.1.

3.5. Using the NEX Allocator From Your Application

You can use the allocator specified in MemoryManager::SetBasicMemoryFunctions by calling new or qNew from a class inheriting from RootObject.

Code 3.1 Sample Use of NEX Allocator

// `NATTraversalClient` inherits from `RootObject.` Therefore:
// The following code uses the allocator configured by `MemoryManager::SetBasicMemoryFunctions.`
NATTraversalClient* pNATTraversalClient = qNew NATTraversalClient();
qDelete pNATTraversalClient;

3.6. Speeding Up Memory Allocation

One factor that may slow down the CTR CPU when using NEX is the extremely high frequency at which NEX allocates and deallocates memory.

For this reason, we recommend using a multi-unit heap as a method to reduce the cost of memory allocation and deallocation. A multi-unit heap consists of multiple unit heaps, and serves as a high speed memory allocator that allocates the requested memory size from the most suitable unit heap available for allocation. Be careful when using a multi-unit heap, because the amount of wasted memory is greater than when using the normal memory allocator.

A multi-unit heap NEX sample program has been released. You may make changes to the multi-unit heap sample program freely, and you may embed it in your application. For more information, see the sample program.

3.7. Guide for the Memory Required When Using Server Features

The following recommendations apply to the memory required for the transport-thread mode. However, the guide for memory required when using the NEX server functions differs according to the functions used and how often they are called.

  • During the NgsFacade::Login function's asynchronous processing: Maximum of 260 KB
  • Continuously calling only the Scheduler::Dispatch function without running asynchronous processing during login (when in a static state): Approximately 214 KB
  • Matchmaking: Assuming that asynchronous processes are not called multiple times simultaneously, it operates without problems if about 64 KB is added to the memory required when in a static state. (Excludes transition processing to peer-to-peer sessions during P2P communications.)
  • Ranking: Depends primarily on the amount acquired with the RankingClient::GetRanking function. When 20 are acquired, 34 KB is added to the memory used when in a static state. When the maximum of 255 is acquired, about 472 KB is added

However, when receiving a notification or a message, a maximum of about 2 KB of memory is allocated when calling the reception callback. (This memory is released when the callback completes.) For this reason, if a large number of notifications or messages are received at the same time, a large amount of memory is consumed. Accordingly, to make sure you allocate enough memory, consider the worst-case scenario when using notifications or messaging. Then, in addition to this memory, allocate about 20% extra memory for emergencies. We also recommend performing immediate logout processing when more memory than anticipated is consumed.

Because the amount of memory used during P2P communications varies widely depending on the number of devices communicating and the functions used, you must insert code in the application to measure memory consumption. This code helps to clarify how much memory is being consumed when in a static state, and how much variation there is.

3.8. Managing Packet Buffer Memory

Within NEX memory allocation, the packet buffer manager performs memory management for P2P communication.

The packet buffer manager is a feature that allocates a dedicated memory pool (the packet buffer pool) as a batch process before starting P2P communication, and then allocates memory from that packet buffer pool according to packet buffer memory allocation requests.

By default, a packet buffer memory manager that uses a packet buffer pool is enabled.

Memory allocation for the packet buffer pool is allocated with the same method as other NEX objects.

Because packet buffer pools are allocated with a predetermined size limit, this can prevent the entire NEX memory from being depleted by memory consumption when sending or receiving a large packet.

The packet buffers used with the following communications are not managed with the packet buffer manager but have the same memory management as other NEX objects.

  • Communication with the server.
  • Automatic communication by the NEX library using duplicated objects.
  • Communication using core duplicated objects.
  • Sending and receiving data in excess of the maximum size that the packet buffer manager can allocate (1364 bytes).

For more information about duplicated objects, see Basic Duplicated Object Features.

Warning

Sending large files affects the memory usage of the NEX library as a whole, because large files are not managed by the packet buffer manager.

3.8.1. Disabling the Packet Buffer Pool

If you want to manage the memory for packet buffers the same way you manage memory for other NEX objects, without using the packet buffer pool, pass false to the PacketBufferManager::EnablePacketBufferPool function.

Set this value before starting P2P communication.

You must set this value every time you initialize the NEX library. The set values are maintained until NEX terminates.

Code 3.2 Example of Managing Memory for Packet Buffers the Same Way as Other NEX Objects

qBool ret = PacketBufferManager::EnablePacketBufferPool(false);
if (!ret)
{
    // Not possible to set during P2P communication; it will return an error in this case.
}

3.8.2. The Packet Buffer Pool

The packet buffer pool is used to allocate memory for the packet buffers used for P2P communication.

The packet buffer pool is allocated before starting P2P communication and is deallocated when P2P communication stops. Specifically, memory is allocated when the ConnectivityManager::StartNATSession, NetZ::NetZ, or VSocket::Open function is called, and deallocated when the ConnectivityManager::StopNATSession, NetZ::~NetZ, or VSocket::Close function is called.

The packet buffer pool is managed by using multiple unit heaps and is used to allocate objects used for managing packets attached to payloads that are not packet payloads. Of these, for the payload unit heap, the unit size and number of memory blocks can be customized on the application side. Customize according to the application communication data size.

Packet buffer allocation attempts memory allocation from the unit heap with the smallest unit size, and uses the first unit heap that can be allocated. If the maximum size also cannot be allocated on the unit heap, the packet buffer pool will be depleted. (Data larger than 1364 bytes cannot be allocated using the packet buffer pool.)

The default packet buffer pool uses a unit heap with a unit size of 1364 bytes for the payload and a 300 kilobyte region, including a unit heap for the packet management object.

3.8.3. Customizing the Unit Heap Structure

You can configure the structure of the payload unit heap using the PacketBufferManager::SetUnitHeapParam function. The NEX library automatically configures the unit heap for the packet management object in accordance with the structure of the payload unit heap.

Configure the unit heap structure before starting P2P communication. You cannot configure it during P2P communication. You do not need to configure it again for every P2P session. Once you have configured the unit heap structure, the NEX library maintains this setting until it is finalized.

All of the following conditions must hold for the unit-heap structure.

  • The unit size is a multiple of 4 and no larger than 1364 bytes.
  • It has a unit heap with a maximum unit size (1364).

A unit heap with the maximum size is required for features such as NEX-internal communication and packet bundling (which is used to send data more efficiently). Make sure that there are at least 64 memory blocks. If this is not enough memory, increase the size.

Warning

  • Set the unit size about 80 bytes larger than the size of data your application will send (while remaining a multiple of 4), because the NEX library also uses this memory for things like headers.
  • The PacketBufferManager::SetUnitHeapParam function ends in an assert error if the unit-heap structure setting is invalid.

Code 3.3 Example of Configuring Unit Heap Structure

qVector<PacketBufferManager::UnitHeapParam> param(3);
// For short-size packets.
paramList[0].unitSize = 128+80; // The extra 80 bytes is to account for use by NEX in headers, and so on.
paramList[0].unitNum = 32;
// Size mainly used by the application + 64.
paramList[1].unitSize = 500+80; // The extra 80 bytes is to account for use by NEX in headers, and so on.
paramList[1].unitNum = 64;
// Packet size for packet bundling (using the maximum of 1,364).
paramList[2].unitSize = 1364;
paramList[2].unitNum = 128; // Get more memory.
qBool ret = PacketBufferManager::SetUnitHeapParam(param);
if (!ret)
{
    // It is not possible to set this during P2P communication. In that case, it returns an error.
    // It ends in an assert error if the unit-heap structure is invalid.
}

This example demonstrates how to define the structure by setting the fields of the PacketBufferManager::UnitHeapParam instance individually. In addition to this method, you can also get a simple configuration from the PacketBufferManager::GetSimpleUnitHeapParam function.

Specify the size of the packet buffer pool and the average size of data sent to the PacketBufferManager::GetSimpleUnitHeapParam function. Simple structures are calculated to fit in a size of a packet buffer pool for which the memory totals for the unit heaps for the payload and for the packet management objects were specified. Of the memory used by the unit heap for payloads, half is allocated as a unit heap the size of the average send data, and the remaining half is allocated as an unit heap of the maximum size (1,364). Note that the unit size takes the header attached inside NEX, and the average send data send size is increased by 80 and is revised to a value that is a multiple of 4.

Code 3.4 Example of Simple Structure Configuration

// The pool size is 500 KB, and the average size of sent data is 500 bytes.
qVector<PacketBufferManager::UnitHeapParam> param;
qBool ret = PacketBufferManager::GetSimpleUnitHeapParam(500*1024, 500, param);
if (!ret)
{
    // If the pool size is too small, the average size of sent data will be invalid, resulting in an error.
}
else
{
    ret = PacketBufferManager::SetUnitHeapParam(param);
    if (!ret)
    {
        // It is not possible to set this during P2P communication. In that case, it returns an error.
        // It ends in an assert error if the unit-heap structure is invalid.
    }
}

3.8.4. Getting Statistical Information

You can get statistical information about allocation of memory for the payload unit heap by using the PacketBufferManager::GetUnitHeapDebugInfo function. You can get the number of memory blocks used, the number of memory allocation failures, and other information, and use this information to inform how you configure the unit heap structure, and check for buffer depletion.

In NEX, a check is performed if there is sufficient availability before buffer allocation; when there is an insufficiency, it may be counted as an occurrence of memory depletion. The count for memory depletion occurrences may increase without the peak value of the actual allocated number reaching the maximum.

Warning

When the memory depletion occurrence count for the unit heap with the largest unit size increases, set the unit heap structure so that the maximum for the number of memory blocks is increased.

Code 3.5 Example of Checking for Packet Buffer Depletion

qVector<PacketBufferManager::UnitHeapDebugInfo> debugInfo;
PacketBufferManager::GetUnitHeapDebguInfo(debugInfo);
qVector<PacketBufferManager::UnitHeapDebugInfo>::const_iterator it;
it = debugInfo.begin();
while (it != debugInfo.end())
{
    // Display the unit size of each unit heap, and the number of times memory has been depleted for each.
    QLOG(EventLog::Info, _T("unitSize:") << it->unitSize << _T(" outOfMemory:") << it->outOfMemory);
}

3.8.5. Operations When the Packet Buffer Pool Is Depleted

The following occurs when the packet buffer pool is depleted.

  • Packets received from other stations are dropped.
  • Calls to functions that send data (such as RMC and direct streaming) fail.

When packets received from other stations are dropped, it may cause NAT traversal between stations to fail, or connections between stations to be lost because of keep-alive timeouts.

QERROR(Transport, PacketBufferFull) is returned when the packet buffer does not have enough space for a sending function. If the function returns an error, wait for a while, and resend as necessary.

If the packet buffer frequently experiences insufficient space, try one or more of the following: allocate more memory to the packet buffer pool, use reliable communication less frequently, or reduce the maximum number of sending buffers for reliable communication.

For the setting of the maximum number of reliable send buffers, see the following documentation.

3.9. Rough Guide for Dynamic Memory and Packet Buffer Usage During P2P Communication

3.9.1. Rough Guide for Maximum Dynamic Memory and Packet Buffer Usage

The maximum amount of memory and packet buffer space required by NEX depends on the connection status, and the frequency with which you send or receive data. The following section is a rough guide to how much memory is required.

Table 3.1 lists the matchmaking conditions. Table 3.2 shows the dynamic memory and packet-buffer usage under these conditions, when the connection status is modified by using the packet delay and loss emulator.

The dynamic memory usage uses a 32-byte alignment memory allocator between the points:

  1. NEX initialization
  2. Start of matchmaking
  3. End of matchmaking
  4. NEX termination

to measure the peak active memory and packet buffer usage.

It shows the packet-buffer usage as eighty memory blocks of 580 bytes, and 64 memory blocks of 1364 bytes.

Table 3.1 Matchmaking Conditions for Investigation of Memory Usage
Item Setting Remarks
Thread mode Core::ThreadModeUnsafeTransportBuffer  
Number of devices joining matchmaking 4  
Matchmaking time 3 minutes This time is the time from when all participants joined the session, until they left it.
Reliable communication frequency 12 per second Sent 500 bytes of data to all participants each transmission. (Use DirectStream.)
Unreliable communication frequency 60 per second Sent 500 bytes of data to all participants each transmission. (Use DirectStream.)
Size of the packet buffer pool 200 KB Using a unit heap structure with 80 memory blocks of 580 bytes, and 64 of 1364 bytes.
Use of auto-stacking Not used. Set Core::UseThreadAutoStack(false).
Table 3.2 Rough Guide to Memory Usage in Development Builds
Packet Delay or Loss Maximum Memory Usage Packet Buffer Usage Remarks
Good environment — almost no packet delays or loss. Approximately 800 KB 25/80, 15/64  
Degraded environment — packet delay of 200 milliseconds, packet loss of 20%. Approximately 800 KB 80/80, 60/64 Matchmaking fails, stations become disconnected from the game server, and send errors occur. In some cases, there are not enough packet buffers.
Packet delay of 200 milliseconds, packet loss over 20%. Approximately 650 KB 2/80, 2/64 At this usage, the loss rate is high, and it is not possible to set up a P2P session.
Table 3.3 Rough Guide to Memory Usage in Release Builds
Packet Delay or Loss Maximum Memory Usage Packet Buffer Usage Remarks
Good environment — almost no packet delays or loss. Approximately 780 KB 25/80, 15/64  
Degraded environment — packet delay of 200 milliseconds, packet loss of 20%. Approximately 780 KB 80/80, 60/64 Matchmaking fails, stations become disconnected from the game server, and send errors occur. In some cases, there are not enough packet buffers.
Packet delay of 200 milliseconds, packet loss over 20%. Approximately 620 KB 2/80, 2/64 At this usage, the loss rate is high, and it is not possible to set up a P2P session.

In the example of a development build:

More than 800 KB of memory is required to run the application, even in a degraded environment (packet delay of 200 milliseconds, packet loss of 20%).

As an example of how to deal with low memory conditions, the application could allocate slightly more memory than needed (say, 900 KB), and shut down the library if memory usage reaches 850 KB.

In a degraded environment (packet delay of 200 milliseconds, packet loss of 20%), it is likely that the library will run out of packet buffers. If you have spare memory, you can reduce the likelihood of running out of packet buffers in these conditions by adding an additional 20-100 KB or so to the packet buffer pool.

3.9.2. Automatic Distribution of Dynamically Allocated Memory

The maximum amount of memory required by NEX depends on the communication environment, and the frequency with which you send or receive data. The following section is a rough guide to how much memory is required. To reduce variation in the amount of memory used for P2P communication depending on the connection conditions, use a packet buffer pool.

The amount of memory NEX requires tends to skew toward a particular amount. You can reduce the CPU load required to allocate NEX memory by using a unit heap, for example, for high-speed memory allocation of frequently requested amounts of memory.

Note

A unit heap wastes more memory than a standard heap. Even if you use a unit heap, we recommend that you use a standard heap to allocate amounts of memory above a particular value (that is, memory greater than 4096).

Table 3.5 and Table 3.6 give rough estimates of the amount of allocated memory distributed under the matchmaking conditions listed in Table 3.4, assuming best-case network conditions where there is almost no packet loss or packet latency. Refer to this information for faster memory allocation.

The dynamic memory usage uses a 32-byte alignment memory allocator between the points:

  1. NEX initialization
  2. Start of matchmaking
  3. End of matchmaking
  4. NEX termination

to measure the memory allocation size distribution.

Table 3.4 Matchmaking Conditions Used in the Dynamic Memory Allocation Study
Item Setting Remarks
Thread mode Core::ThreadModeUnsafeTransportBuffer  
Number of devices joining matchmaking 4  
Matchmaking time 3 minutes This time is the time from when all participants joined the session, until they left it.
Reliable communication frequency 12 per second Sent 500 bytes of data to all participants each transmission. (Use DirectStream.)
Unreliable communication frequency 60 per second Sent 500 bytes of data to all participants each transmission. (Use DirectStream.)
Size of the packet buffer pool 200 KB Using a unit heap structure with 80 memory blocks of 580 bytes, and 64 of 1364 bytes.
Use of auto-stacking Not used. Set Core::UseThreadAutoStack(false).
Table 3.5 Rough Estimates of the Distribution of Dynamically Allocated Memory Sizes in Development Builds
Requested Size Estimate of the Maximum Number of Simultaneous Allocations Estimate of the Total Number of Allocations Remarks
1 to 32 bytes 820 96000  
33 to 64 bytes 540 53000  
65 to 128 bytes 120 1400  
129 to 256 bytes 90 400  
257 to 512 bytes 70 20000  
513 to 1024 bytes 180 11000  
1025 to 1536 bytes 160 50000  
1537 to 2048 bytes 6 8  
2049 to 4096 bytes 17 100  
4097 to 8192 bytes 4 4  
32,767 bytes 3 3 Up to three 32 KB blocks (32 KB - 1 byte) of memory are always allocated for the thread stack by Core::ThreadModeUnsafeTransportBuffer.
200 KB 1 1 Used by the packet buffer pool.
Table 3.6 Rough Estimates of the Distribution of Dynamically Allocated Memory Sizes in Release Builds
Requested Size Estimate of the Maximum Number of Simultaneous Allocations Estimate of the Total Number of Allocations Remarks
1 to 32 bytes 820 96000  
33 to 64 bytes 540 52000  
65 to 128 bytes 120 1300  
129 to 256 bytes 70 400  
257 to 512 bytes 60 20000  
513 to 1024 bytes 180 11000  
1025 to 1536 bytes 160 50000  
1537 to 2048 bytes 6 8  
2049 to 4096 bytes 17 100  
4097 to 8192 bytes 4 4  
32,767 bytes 3 3 Up to three 32 KB blocks (32 KB - 1 byte) of memory are always allocated for the thread stack by Core::ThreadModeUnsafeTransportBuffer.
200 KB 1 1 Used by the packet buffer pool.

CONFIDENTIAL