5. NEX Library (General) (Basic Types, Dispatching, Asynchronous Processing, and Data Integrity)

5.1. Basic Types Defined by NEX

NEX independently defines basic types, such as int, but you can also directly use the basic types defined in the SDK. Table 5.1 shows the correspondence between the basic types defined by NEX and the basic types defined in the SDK.

Table 5.1 Correspondence Between Basic Types Defined by NEX and the SDK
Names of Types Defined by NEX Corresponding Names in SDK
nn::nex::qBool bool
nn::nex::qByte u8
nn::nex::qInt8 s8
nn::nex::qUnsignedInt8 u8
nn::nex::qInt16 s16
nn::nex::qUnsignedInt16 u16
nn::nex::qInt32 s32
nn::nex::qUnsignedInt32 u32
nn::nex::qInt Int
nn::nex::qUnsignedInt unsigned int
nn::nex::qFloat f32
nn::nex::qDouble f64
nn::nex::Real f32
nn::nex::qInt64 s64
nn::nex::qUnsignedInt64 u64
nn::nex::qChar8 char8
nn::nex::qChar16 char16
nn::nex::qWideChar char16
nn::nex::qChar char16

5.2. Overview of Dispatch Processing

NEX is designed to perform internal processing and send and receive packets by having applications periodically execute the Scheduler::Dispatch function. For both P2P communications and server communications, you should periodically call the Scheduler::Dispatch function after a NgsFacade or NetZ object has been created, and until it is later destroyed. However, because the Scheduler::Dispatch function is called inside the CallContext::Wait function, you must not call Scheduler::Dispatch until after the process has returned from CallContext::Wait.

The Scheduler::DispatchAll member function dispatches all jobs in the dispatch queue for processing. This member function exits when all jobs to dispatch are finished.

Meanwhile, you can specify a timeout for the Dispatch member function. In this case, dispatching continues until the specified timeout is reached, or the dispatch queue is empty of jobs. This member function specifies the timeout set using the uiDispatchTimeout parameter in milliseconds. The default value is infinite, and equivalent to the Scheduler::DispatchAll function. This function returns the number of jobs that could not be dispatched. Continually returns a value of 1 or higher. If the return value is getting larger, this means the timeout for processing is not long enough. Either increase the timeout or reduce the amount of data being sent.

The Scheduler::GetInstance function returns a pointer to a Scheduler object.

A Scheduler object is created within the NEX library when an NgsFacade object or NetZ object is created. Use the Scheduler::GetInstance function to get this Scheduler object. Call the Dispatch function periodically while a Scheduler object exists. The Scheduler object is deleted automatically along with the NgsFacade object or NetZ object.

5.2.1. Example of Recommended Usage of Dispatching to Minimize Communication Latency

This section describes the recommended usage of dispatching to minimize communication latency.

  1. Execute the DispatchAll or Dispatch function (for receiving).
  2. The application handles the received data. If you are using duplicated objects, refresh all duplicated object duplicas to apply the received data.
  3. Change the game state (such as physics, AI, and so on).
  4. Set the data to send in the objects. If you are using duplicated objects, update all duplication masters to prepare to send data.
  5. Execute the DispatchAll or Dispatch function (for sending).
  6. The application performs rendering and other operations.

If you use NetZ duplicated objects, the DispatchAll function or Dispatch function is executed in each frame. You must download the latest data for all duplicated objects before updating (or refreshing) a duplicated object dataset. It is also possible that the DispatchAll function or Dispatch function can be executed just one additional time per frame. If this happens, it causes some latency.

In cases where the P2P features of NetZ are not being used, for example, when only using rankings or data stores, if there are intervals of up to around 100 msec, you can increase the interval between dispatch calls.

5.3. Asynchronous Processing

To run asynchronous processing with NEX — while communicating with a server, for example — pass a CallContext class or its subclass ProtocolCallContext (when there is communication with a server) as an argument to an asynchronous function. You can also get information related to the CallContext state and asynchronous processing results, cancel or reset the CallContext, set timeouts, and so on.

The State and Outcome of the CallContext object change when asynchronous processing ends. State has one of five values: Available, CallInProgress, CallSuccess, CallFailure, and CallCancelled. It is initially set to Available. The state changes to CallInProgress when asynchronous processing is invoked, and afterwards changes to CallSuccess, CallFailure, or CallCancelled. When a call context is in the CallInProgress state, it can be canceled by the CallContext::Cancel member function and the state changes to CallCancelled. When the state is CallSuccess, CallFailure, or CallCancelled, you can call the CallContext::Reset function to reset the state to Available. The CallContext::GetState function returns the current state of the CallContext object. When the CallContext::Wait function is used, thread execution waits until asynchronous processing ends. You can use the CallContext::GetOutcome function to get the asynchronous processing result.

_images/Fig_Overall_CallContext.png

Figure 5.1 CallContext State Transition Chart

You can cause a user-defined callback to be invoked at the same time that a CallContext object completes. By using a completion callback, you avoid continuously polling CallContext after asynchronous processing is invoked to determine whether processing has completed. The callback is registered with a specific CallContext by the RegisterCompletionCallback method.

void CallContext::RegisterCompletionCallback(
                              CompletionCallback pfCompletionCallback,
                              const UserContext & oContext,
                              qBool bAddToEnd=true);

void CallContext::RegisterCompletionCallback(
                             CallbackRoot* pCallback,
                             qBool bCallOnSuccess=true,
                             qBool bAddToEnd=true);

This function takes either a pointer to the CompletionCallback function (pfCompletionCallback) that is invoked when the CallContext completes, or a pointer to a CallbackRoot object. If you choose to use the second signature, you must define your CallbackRoot object using the Callback template class. A UserContext object is passed as an argument when pfCompletionCallback is run.

5.3.1. Asynchronous Processing Errors

You can get asynchronous processing results with the CallContext::GetOutcome function. If an error occurs, get the qResult using the CallContext::GetOutcome function. qResult contains two types of macros: QRESULT_SUCCESS and QRESULT_ERROR, which generate the qResult class that includes the specified error value; and QSUCCESS and QERROR, which represent the error value itself. QSUCCESS and QERROR are faster than QRESULT_SUCCESS and QRESULT_ERROR when comparing errors.

If the error is on the list of errors that the application should handle, use the application to perform the appropriate error handling. For more information, see the API Reference, or the sections on error handling for the various features.

If an undocumented error occurs, use the following code to get the network error code with the ErrorCodeConverter::ConvertToNetworkErrorCode function, pass the error code to the error/EULA applet, and then shut down communication (Code 5.1).

5.3.2. List of Errors for the Application to Handle

  • Matchmaking
    • QERROR(RendezVous, SessionVoid) The gathering does not exist.
    • QERROR(RendezVous, SessionFull) The maximum number of participants has already been reached.
    • QERROR(RendezVous, PersistentGatheringCreationMax) The registration limit for the persistent gathering has been exceeded.
    • QERROR(RendezVous, PersistentGatheringParticipationMax) The limit on persistent gathering participants has been exceeded.
    • QERROR(RendezVous, InvalidGatheringPassword) The password does not match the persistent gathering password.
    • QERROR(RendezVous, WithoutParticipationPeriod) The session is not within the available participation period.
    • QERROR(RendezVous, DeniedByParticipants) Matchmaking session participants have the local station registered on their blacklist.
    • QERROR(RendezVous, ParticipantInBlackList) A participant in the matchmaking session is registered in the local blacklist.
    • QERROR(RendezVous, NotParticipatedGathering) The specified user has not joined the gathering.
    • QERROR(RendezVous, MatchmakeSessionUserPasswordUnmatch) The matchmaking session user password does not match.
    • QERROR(RendezVous, MatchmakeSessionUserPasswordUnmatch) The matchmaking session system password does not match.
    • QERROR(RendezVous, UserIsOffline) The specified user is offline.
    • QERROR(RendezVous, NotFriend) The specified user is not a friend.
    • QERROR(RendezVous, SessionClosed) The gathering is closed.
    • QERROR(MatchmakeReferee, InvalidArgument) An argument is invalid.
    • QERROR(MatchmakeReferee, AlreadyExists) Personal statistics already exist.
    • QERROR(MatchmakeReferee, NotParticipatedGathering) Not a participant in the gathering.
    • QERROR(MatchmakeReferee, NotParticipatedGathering) Not a participant in the round.
    • QERROR(MatchmakeReferee, StatsNotFound) Statistical data was not found.
    • QERROR(MatchmakeReferee, RoundNotFound) The round was not found.
    • QERROR(MatchmakeReferee, RoundArbitrated) The round has been summarized.

You can choose to either handle the following error message in the application or pass it to the error/EULA applet.

  • QERROR(RendezVous, GameServerMaintence) Game server maintenance will begin.
  • QERROR(RendezVous, InvalidOperation) Attempted an invalid operation.
  • QERROR(RendezVous, AlreadyParticipatedGathering) Already joined the gathering.
  • QERROR(RendezVous, PermissionDenied) Attempted an operation without having the permission to do so.
  • Ranking
    • QERROR(Ranking, NotFound) Data not found.
  • Data Stores
    • QERROR(DataStore, NotFound) Data not found in specified data ID or persistence slot.
    • QERROR(DataStore, PermissionDenied) Attempted operation without password on data to which user does not have privilege.
    • QERROR(DataStore, UnderReviewing) Access denied because data is under review.
    • QERROR(DataStore, InvalidPassword) Attempted operation with incorrect password.
    • QERROR(DataStore, ValueNotEqual) Could not update because values are not equal.
  • P2P Communication Features

With the following two error messages, it might be possible to continue communications if the application retries sending of takes some other measure. However, if these errors occur during application development, consider such improvements as reducing the frequency at which data is sent, increasing the number of times you poll for the reception of data, altering the buffer size, or decreasing the number of Wi-Fi access points when there are too many in the vicinity.

  • QERROR(Transport, ReliableSendBufferFull) Could not send because the buffer for DirectStream reliable communication is full.
  • QERROR(Transport, PacketBufferFull) Could not send due to packet buffer exhaustion.

For the following error message, perform the error handling that is appropriate for the situation.

  • QERROR(Transport, InvalidStation) The destination Station does not exist. (It is either connecting or never existed.)

The following four error messages can occur when using duplicated objects if a station is leaving a session or the communication environment worsens. The application can continue communications with other stations by ignoring these error messages.

  • QERROR(DOCore, StationNotReached) Cannot reach the station being called.
  • QERROR(DOCore, TargetStationDisconnect) The call was executed, but the connection with the target station was cut before a response was returned.
  • QERROR(DOCore, LocalStationLeaving) The call was executed, but the local station is leaving the session so the response will not be received.
  • QERROR(DOCore, CallTimeout) The call failed because the process could not be completed within the configured timeout time.
  • QERROR(DOCore, MigrationInProgress) Object migration is in progress.

Code 5.1 Sample Error Handling

MatchmakeExtensionClient client;
ProtocolCallContext context;

// Error handling for function calls.
if(!client.JoinMatchmakeSession(&context, gid, _T("join message")))
{

    // A CallContext was passed in while it was still being processed.
    NN_ASSERT(context.GetState() != CallContext::CallInProgress);

    // Bind has not been run.
    NN_ASSERT(client.GetCredentials() != NULL);



    // Other, such as the network is not connected.
    // Display the message "A communication error occurred." (The error code is unnecessary.)
    // Log out and shut down.
}

// Error handling for asynchronous processing.
context.Wait(); //Block and wait here.


if(context.GetState() != CallContext::CallSuccess)
// WARNING: To cancel context, a comparison with CallContext::CallCancelled is also required.
{
    qResult result = context.GetOutcome();

    // Errors that must be handled by the application.
    if(result == QERROR(RendezVous, SessionVoid))
    {
        // There is no matchmaking session.
    }
    else if(result == QERROR(RendezVous, SessionFull))
    {
        // The matchmaking session is full.
    }
    else
    {
        u32 code = ErrorCodeConverter::ConvertToNetworkErrorCode(result);
        // Start the error/EULA applet.
        // Log out and shut down.
    }
}

5.4. API Functions That May Block for Extended Periods

Note that any API without a CallContext (except for functions that access or update local values or those that perform simple processing, such as ID conversion) could block for extended periods at any time.

When a blocking operation occurs with a NEX API called from a user thread, the Scheduler::Dispatch function automatically executes until the blocking operation finishes. A NEX API returns an error if a NEX API that requires another blocking operation is called by a callback called from that Scheduler::Dispatch function. This helps avoid deadlocks caused by blocking waits and the possibility that jobs cannot be executed in order because the Scheduler::Dispatch function was called twice.

The following list shows API functions that may block, and the conditions under which they block. It does not describe NEX API functions that can run asynchronously using a CallContext instance.

Table 5.2 API Functions That May Block and Conditions for Blocking
API Function Conditions for Blocking
DuplicatedObject::Create If called by a station other than the session master, it may block while communicating with the session master to generate an ID.
DuplicatedObject::Publish If you called the DuplicatedObject::Create function before joining the session, the DuplicatedObject::Publish function may block communication with the session master to generate an ID.
DOCallContext::Wait It blocks if the object this DOCallContext refers to is not yet discovered, or if the call has not yet completed.
DORef::Wait It blocks if this DORef references a duplicated object that has not been discovered yet.

5.5. Cautions for Processing in Callbacks

If you call a blocking operation from a callback, RMC, or from within member functions such as DataSet::OperationBegin, DuplicatedObject::OperationBegin, DataSet::OperationEnd, or DuplicatedObject::OperationEnd, the system returns the SYSTEMERROR_GEN_INVALID_WAIT error when there is a possibility of deadlocking.

In this case, you can use the CallMethodOperation class to ensure that the original calls do not deadlock while waiting for a return value from the blocking operation. This class can delay the call if preparations for processing are not complete. It can call the same member function again after a specified time has elapsed.

You can use the CallMethodOperation::PostponeOperation member function to set the required delay for a callback, RMC, or the DataSet::OperationBegin, DuplicatedObject::OperationBegin, DataSet::OperationEnd, or DuplicatedObject::OperationEnd member functions. For more information about using this class and sample code, see Extended Duplicated Object Features in P2P Communication Features.

5.6. Socket Port Numbers Used With NEX

The CTR uses the following TCP/IP port numbers when using the friend presence feature, server services, and P2P communication.

  • Friend Presence: http, https, UDP 9103
  • Server Services: http, https, UDP 49152–65535
  • P2P Communication: UDP 49152–65535 (A separate port is used for communication with the game server.)

The friend server and game server use the following ports.

  • Friend Server, Game Server: UDP: 50000–60000
  • Game server (NAT traversal server): UDP: 10025, 10125 ,33335
  • Authentication Server and Storage (DataStore): http, https

If you have configured a firewall, enable reception on all UDP ports to allow P2P communication. When connected to a wireless access point with a NAT feature, the address and port of the sender and receiver could change. As a result, NAT traversal fails if only reception on UDP ports 49152–65535 is enabled.

5.7. NEX Communications - Data Integrity

In NEX, to detect data tampering during both server services and P2P communications, packets are given an additional hash value such as a temporary key that differs for each session data payload. "Each session" refers to each time a user logs in to the game server for server services, and refers to each matchmaking session for P2P communications. Also, local communication is unique to each application. You do not need to use CRC data in the application. However, consider the possibility that your application is being hacked, and take measures such as checking the range of data separately.

When data is downloaded from or uploaded to the data store, HTTPS communications are used rather than the communication types described previously. If data tampering occurs over the communication channel, HTTPS communication errors occur. (The API functions return communication errors.)

Also, in NEX, if data that has unexpected hash values is received, those packets are destroyed in the library. Invalid packets are never passed to the application. An ACK packet is not returned to the sender during reliable communication, so the sender may automatically resend data within the library. The result is similar to the result when packets are dropped over the communication channel. Packets cannot be received if tampering occurs, and when the data has been completely spoofed, the API times out, the connection process fails, or the communications are terminated. You do not need to worry about packet tampering over communication channels in the application itself.

5.8. Uniqueness of NEX Server Services IDs

The IDs issued by NEX server services (NEX unique ID, session ID, gathering ID, data ID, and so on) are not unique across game servers. These IDs may be duplicated if the server environment or game server ID is different. For example, the same NEX unique ID that was issued in the development environment might also exist in the Lotcheck environment. If you are using an independent server, do not design it on the assumption that these IDs are unique.


CONFIDENTIAL