nlib
nlib Main Page

This manual is a collection of libraries that support Windows, Linux, FreeBSD, OS X, Cygwin, and Nintendo's development environments.

About nlib

nlib is being developed to provide a group of user-space libraries that enable you to program for Windows, Linux, FreeBSD OS X, Cygwin, and Nintendo's development environments using the same interface. Its development follows a software-centric approach, adopts appropriate and pragmatic coding rules (nlib Coding Rules), and implements an interface that is independent from the individual development environments. This focus makes it possible to develop in parallel for multiple development environments while maintaining high quality, performance, and reliability.

Features Implemented by nlib

The main features implemented are listed below.
  • Reading and writing XML and JSON
  • Highly-efficient memory allocators
  • Test framework compatible with Google Test
  • Wrapper libraries that allows codes to be developed that supports both SSE4.2 and NEON. Includes support for various calculations for 3x4 matrices, and three- and four-dimensional vectors.
  • Succinct data structure libraries for features like keyword searching
  • UTF-8 reading, writing, normalization, and conversion from and to UTF-16 and UTF-32
  • Threads, mutexes, condition variables, TLS, barriers, read-write locks, and atomic variables
  • Getting and freeing system memory, getting the time, and reading and writing files
  • Basic lock-free data structure and algorithms
  • Functions and macro definitions for base C linkages used to provide multi-platform and multi-compiler support

nlib Operating Environment

Currently, nlib has been verified for compilation and operation in the following environments.
Platform Library types Remarks
Windows Visual Studio 2017 64bit static library, DLL, 15.2
Visual Studio 2015 64bit static library, DLL Update 3
32bit static library
Visual Studio 2013 64bit static library, DLL Update 5, Express Edition is available
32bit static library
Cygwin 64bit static library gcc 5.4.0, newlib, libstdc++
Bash on Ubuntu on Windows 64bit shared library Ubuntu 14.04
Linux Ubuntu 17.04 64bit shared library clang 4.0.0, glibc, libstdc++, https://atlas.hashicorp.com/xenji/
Ubuntu 16.04 LTS 64bit gcc 5.3.1, clang 3.8, glibc, libstdc++, https://atlas.hashicorp.com/bento/
Ubuntu 14.04 LTS 64bit gcc 4.8.4, clang 3.4, glibc, libstdc++, https://atlas.hashicorp.com/ubuntu/
32bit
CentOS 7 (v1704.01) 64bit gcc 4.8.5, glibc, libstdc++, https://atlas.hashicorp.com/centos/
Fedora 25 64bit gcc 6.2.1, clang 3.8, glibc, libstdc++. https://atlas.hashicorp.com/fedora/
Alpine 3.5.1/td> 64bit gcc 6.2.1, musl libc, libstdc++, https://atlas.hashicorp.com/maier
FreeBSD FreeBSD 11.0 64bit shared library clang 3.8, BSD libc, libc++, https://atlas.hashicorp.com/bento/
macOS macOS 10.12 (Sierra) 64bit shared library Xcode 8.3.2
CTR CTR-SDK 11.5 32bit static library CTR_SDK-11_5_2-20161105-ja.zip
armcc_4_1_nintendo-b1454-20150227.zip
Cafe CafeSDK 2.13.01 32bit static library cafe_sdk-2_13_01-20150728.zip
MULTI-5_3_26-20150918.zip
Depending on the compiler version and other factors, compilation uses C++11 features as appropriate (C++11 Features Used by nlib).

nlib Support Policy

Please contact the site where the package was released for instructions about how to get support. Also read the documentation, including the FAQ. This document and the code samples explain the current implementation. Future development plans and release dates will not be documented or made public. Support and bug fixes will be made in the latest version at that time. Special versions will not be provided for specific projects, and no special support will be provided. This rule is essential to keep the size of the development team in check.
Unannounced changes that break compatibility may occur if there is no way around it. Note this fact.
Before use, it is probably a good idea to perform some acceptance tests to determine performance criteria including whether the package works as expected, is sufficiently fast, and has memory consumption within the acceptable limits. Creating acceptance tests up front also helps to avoid trouble when upgrading nlib. It is a good idea to use the testing library to create these types of acceptance tests.
The bug may be quickly fixed if you describe the code reproducing the bug as a test code in the nn::nlib::testing format and include it in your bug report.

nlib Package Description

Note that the included libraries and supported platforms vary depending on the package.

Directory Structure

The package is structured as follows.
    ├── bin
    │   └── cmake                 # Command-line tools
    ├── cmake                      # Common Cmake scripts
    ├── documents-en
    │   └── API                   # Reference Manual
    │       ├── html              # HTML version of the Reference Manual
    │       └── searchdata.xml    # Search index
    ├── include                    # Include files (in `system/include` for Cafe)
    │   ├── nn
    │   │   └── nlib
    │   │       ├── exi
    │   │       ├── heap
    │   │       ├── msgpack
    │   │       │   └── jsonrpc
    │   │       ├── oss
    │   │       ├── simd
    │   │       ├── succinct
    │   │       ├── testing
    │   │       ├── threading
    │   │       └── unicode
    │   ├── lz4.h
    │   ├── lz4hc.h
    │   ├── lz4frame.h
    │   ├── sqlite3.h
    │   └── sqlite3ext.h
    ├── lib                        # Platform-specific library files (`system/lib` for Cafe, `libraries` for CTR)
    ├── samples                    # Source code for samples
    │  ├── cmake
    │  └── sources
    │      ├── exi
    │      ├── heap
    │      ├── misc
    │      ├── msgpack
    │      ├── oss
    │      ├── succinct
    │      └── testing
    └── setup
        ├── setup-bash-ubuntu-windows-tools.sh
        ├── setup-cygwin-tools.sh
        ├── setup-msvc-libraries.ps1
        └── setup-osx-tools.sh

Platform-Specific Descriptions

Descriptions of Each Library

libn Library (including libn.a, libn.so, n-vc120amd64.lib, and n-vc120dllamd64.dll, etc.)
This library includes functions that have C linkages for suppressing differences among various platforms.
Specifically, the following functions are provided:
  • Functions, including nlib_epochtime(), that obtain the current time.
  • The nlib_gen_random() function that obtains random values.
  • Allocation and deallocation of (virtual) memory from the operating system
  • Functions and macros supporting pthread
  • Message queues supporting lock-free behavior (nlib_mq)
  • Functions, including nlib_atomic_compare_exchange32(), that atomically rewrite memory.
  • Functions, including nlib_debug_break() and nlib_debug_backtrace(), that access the debugger.
  • The nlib_getenv() function that obtains environment variable values.
  • File handles and file I/O functionality, including nlib_fd_open().
  • Functions relating to string searches, accelerated with SIMD.
  • String encoding conversion between UTF-8 and UTF-16 and between UTF-8 and UTF-32, in both directions
  • Functions, including nlib_strto_int32(), that wrap standard C functions, including strtol() and strtoll(), in a way they are not wrongly used.
  • Functions, including nlib_isspace(), that implement standard C functions defined by ctype.h in a way they are not affected by the locale.
  • Functions, including nlib_snprintf(), that are adjusted to suppress differences among the environment-specific behaviors of snprintf(), snwprintf(), and other similar functions to prevent them from being wrongly used.
  • Fast conversion of endianness (including nlib_swapendian_32())
  • Fast CRC32 and CRC32C calculations (nlib_crc32() and nlib_crc32c())
  • Fast pop count calculations (including nlib_popcnt64())
  • Macros that absorb differences among the various attributes for each compiler
misc Library (libnx_misc.a, libnx_misc.so, nx_misc-vc120amd64.lib, nx_misc-vc120dllamd64.dll, etc.)
This library is a class library that includes relatively small classes. It implements many features that are believed to be commonly used in programs.
Specifically, it includes the following. The namespaces include nn::nlib, nn::nlib::unicode, and nn::nlib::threading.
exi Library (libnx_exi.a, libnx_exi.so, nx_exi-vc120amd64.lib, nx_exi-vc120dllamd64.dll, etc.)
A library used to read and write binary XML. It can also read and write text-based XML. It supports XML namespaces and includes a simple, easily programmable pull parsing API.
The namespace is nn::nlib::exi.
heap Library (libnx_heap.a, libnx_heap.so, nx_heap.lib, nx_heap-vc120dllamd64.dll, etc.)
This library is a heap library for which memory allocation and deallocation is optimized with multithreaded operations. The nmalloc and nfree functions (and the heap::CachedHeap and heap::CentralHeap classes used to implement them) keep performance degradation in check by controlling fragmentation through the use of multi-level free lists. By maintaining per-thread caches for small blocks of memory, memory can be allocated and freed without locking, which helps to limit the degradation of multithreaded performance. Pointers within the free lists are checked for both mangling and their allocation status. This implementation makes it more difficult to use them in an invalid way by overwriting memory externally.
The namespaces are nn::nlib::heap and the global namespace (C linkage functions).
succinct Library (libnx_succinct.a, libnx_succinct.so, nx_succinct-vc120amd64.lib, nx_succinct-vc120dllamd64.dll, etc.)
A library used to handle succinct data structures. You can think of a succinct data structure as one whose content can be searched while it is still compressed.
The succinct library includes the following.
  • Implementation of succinct vector classes
  • A class that implements balanced parentheses
  • A class that implements LOUDS (level-order unary degree sequences)
  • Trie implementation
  • A class for creating high-compression indices and searching strings using the Aho–Corasick algorithm
The namespace is nn::nlib::succinct.
msgpack Library (libnx_msgpack.a, libnx_msgpack.so, nx_msgpack-vc120amd64.lib, nx_msgpack-vc120dllamd64.dll, etc.)
A library for reading and writing MessagePack and JSON. It also includes a class for parsing CSV.
The msgpack library includes the following.
  • Classes for reading and writing JSON from streams
  • Classes for reading and writing MessagePack from streams
  • A class for high-speed in-memory MessagePack parsing
  • A class for reading CSV
  • Classes for storing JSON, MessagePack, and CSV data after it has been read
  • Implementation of JSON-RPC
The namespace is nn::nlib::msgpack.
testing Library (libnx_testing.a, libnx_testing.so, nx_testing-vc120amd64.lib, nx_testing-vc120dllamd64.dll, etc.)
A library for writing highly portable test code. It is a testing framework that is somewhat source-code compatible with tests written for Google Test.
The namespace is nn::nlib::testing. (It is aliased to the testing namespace for the sake of Google Test compatibility.) It depends on the misc library.
oss Library (libnx_oss_bsdiff.a, libnx_oss_sqlite3.a, libnx_oss_lz4.a, etc)
Contains code for open-source software (OSS), code derived from OSS, and code for accessing OSS. To use it, an OSS license must be processed.
The following libraries are currently included. They are built as separate library files.
  • A library conversion of bsdiff that has been modified to use zlib instead of bzlib2, and the corresponding command-line programs
  • A version of SQLite3 that has been modified to run using the interfaces provided by nlib, and a corresponding command-line shell
  • LZ4 is approximately 20 times faster at data compression than zlib, and approximately five times faster at decompression.
The namespaces are nn::nlib::oss and the global namespace (C linkage functions).
simd Library (Mostly Inline Functions Included in the misc Library)
SIMD instructions can significantly increase the speed of programs. The simd library makes it possible to write code that supports both SSE4.2 and NEON.
It supports integer and single-precision floating-point number calculations that enable your code to compute at high speeds.
Single-precision floating-point number calculation can be used even in environments that do not support SSE or NEON.
On Cafe, this is optimized to use paired singles.
In other words, the same code can support the following environments.
  • On x86 processors, the code is compiled with speed boosts through the use of SSE 4.2.
  • On processors with NEON support, the code is compiled with speed boosts through the use of NEON.
  • On Cafe, the code is compiled with speed boosts through the use of paired singles. (Floating-point operations alone are quite a bit slower than the Cafe-SDK for operations like matrix calculations, because the interface returns values for data like four-dimensional vectors.)
  • On CTR, generic code is compiled (floating-point operations only).
The namespace is nn::nlib::simd.

Dependency between nlib libraries.

The dependency between nlib libraries is as shown in the following figure.

dot_inline_dotgraph_32.png

C++11 Features Used by nlib

nlib is compiled using C++11 features for some combinations of compilers and versions. The table below summarizes the C++11 features, compatibility macros that can be used with nlib, and the compilers that compile using C++11. Note that for clang, the determination is made based on __has_feature and __has_include macros, and not the version.
C++11 features are not used in any other environment.
C++11 Feature Identification Macro Macros and Typedefs for C++03 Compatibility Visual Studio gcc
Rvalue references NLIB_CXX11_RVALUE_REFERENCES None 2010 4.3
Variadic templates NLIB_CXX11_VARIADIC_TEMPLATES None 2013 4.7
Static assertions NLIB_CXX11_STATIC_ASSERTIONS NLIB_STATIC_ASSERT() 2010 4.3
Null pointer constant NLIB_CXX11_NULL_POINTER_CONSTANT nn::nlib::nullptr, nn::nlib::nullptr_t 2010 4.6
Deleted functions NLIB_CXX11_DEFAULTED_AND_DELETED_FUNCTIONS NLIB_DISALLOW_COPY_AND_ASSIGN() 2015 4.4
Explicit conversion operators NLIB_CXX11_EXPLICIT_CONVERSION_OPERATORS NLIB_SAFE_BOOL() 2013 4.6
Alias templates NLIB_CXX11_TEMPLATE_ALIAS None 2013 4.7
constexpr NLIB_CXX11_CONSTEXPR NLIB_CEXPR 2015 4.7
char16_t, char32_t NLIB_CXX11_NEW_CHARACTER_TYPES nlib_utf16_t, nlib_utf32_t 2015 4.4
alignas NLIB_CXX11_ALIGNMENT_SUPPORT NLIB_ALIGNAS() 2015 4.8
Explicit virtual overrides NLIB_CXX11_EXPLICIT_VIRTUAL_OVERRIDES NLIB_OVERRIDE 2012 4.7
final NLIB_CXX11_EXPLICIT_VIRTUAL_OVERRIDES NLIB_FINAL 2012 4.7
noexcept NLIB_CXX11_NOEXCEPT NLIB_NOEXCEPT 2015 4.6
Range-based for NLIB_CXX11_RANGE_BASED_FOR None 2012 4.6
lambda NLIB_CXX11_LAMBDAS None 2012 4.7
Delegating constructors NLIB_CXX11_DELEGATING_CONSTRUCTORS None 2013 4.7
std::unique_ptr NLIB_CXX11_UNIQUEPTR nn::nlib::UniquePtr 2013 4.7
#include <random> NLIB_CXX11_STDLIB_RANDOM None 2013 4.8
#include <chrono> NLIB_CXX11_STDLIB_CHRONO Conversion to and from nn::nlib::DateTime 2012 4.7
#include <atomic> NLIB_CXX11_STDLIB_ATOMIC NLIB_MEMORY_ORDER_SEQ_CST and others 2012 4.8
#include <array> NLIB_CXX11_STDLIB_ARRAY None 2012 4.5
#include <unordered_map>
#include <unordered_set>
NLIB_CXX11_STDLIB_UNORDERED None 2012 4.5
#include <tuple> NLIB_CXX11_STDLIB_TUPLE None 2012 4.5

nlib Coding Rules

nlib Basic Coding Rules

The nlib coding rules are a slight modification and relaxation of the latest version of the Google C++ Style Guide (and a slightly outdated Japanese translation).
The reasons for using the Google C++ Style Guide are listed below.
  • It is easy to access documentation about the coding rules. Translations are available (albeit unofficial ones).
  • Similar to the general coding rules for the gaming industry, C++ exceptions and RTTI use are prohibited.
  • It has been updated continuously in the past.
  • Changes made to the coding rules with each update are reasonable, and there haven't been any sudden, drastic changes.
  • We can expect it to be maintained even five years from now.
The changes and relaxations adopted for nlib are shown below.
Coding rule Google C++ Style Guide nlib
Line length 80 100
Indentation 2 4
Parameter order Inputs followed by outputs Outputs followed by inputs
Default argument Prohibited in principle Allowed for use except for the cases described below:
  • In the case of a constructor. Issues may arise if the default value allows an argument not to be explicitly specified until only one argument remains unhandled.
  • In the case of a virtual function. Issues may arise if the overridden function has a default argument different from that of the original function.
  • In the case where the argument can be used as a function pointer. Confusion may be induced in the coding.
File name file_name.cc FileName.cpp(file_name.cpp in the sample)
#include guard Also append #pragma once (for accelerating compilation in Visual Studio)
Use of boost Not to be used by libraries (may be used for other purposes such as testing)
Accessor function name get_xyz() / set_xyz() GetXyz() / SetXyz()
Argument passed by reference Non-const references are prohibited In nlib, non-cost references are allowed as exceptions in the following cases.
  • When specializing std::swap
  • Code that imitates move semantics in C++03
  • Class with simple features. For example, a class having a feature that only maintains a lock, such as the ScopedLock class.
  • When passing nn::nlib::UniquePtr as an argument (Returning a value would be inefficient, and it would be strange to pass a pointer to UniquePtr.)
Use of C++11 Ensure that headers and source code can be properly compiled also by using C++03 compilers.
  • This doesn't mean you can't use C++11. Features such as move constructors, range-based for statements, explicit virtual overrides, and explicit conversion operators should all be supported proactively.
  • If possible, the same source code is compiled using macros and other preprocessor directives. If not possible, implement similar features that can be used from C++03.
nullptr Use if C++11 In principle, don't use nullptr_t or nullptr as long as C++03 is used.
  • The nullptr idiom for implementing nullptr in C++03 results in errors if it is defined in multiple libraries, so it cannot be defined in the global namespace.
  • With nlib, it is defined as nn::nlib::nullptr_t and nn::nlib::nullptr for C++03.
  • Avoiding the use of misleading overloads is a workaround for most cases that would require nullptr.
constexpr Use if C++11 For the time being, use of constexpr should be avoided except in obvious cases.
  • Unfortunately, code that uses constexpr proactively has a drawback in terms of portability, because differences among compiler behavior can result in many compilation errors.
  • "Obvious cases" refers to null constant objects or cases that one would expect to compile without issues on any compatible compiler that supports constexpr (for example, simple constructors that merely set constants).
Comment
  • Considering translation costs, do not use non-ASCII (non-English) characters in comments for public headers (This rule does not apply if the content should be never translated).
  • If you want to write detailed comments, include them in either source code files or the Reference Manual.
  • Doxygen comments must not be written in public headers. This causes recompilation due to changes in the documentation, and increases the translation burden. Files such as manual_ja.h and manual_en.h should be prepared for each library and comments should be written into those files.
  • In the reference manuals, never place newline characters in the middle of a sentence. This rule is meant to support the nature of the translation-aid software used for localization. It takes precedence over the previously mentioned limit of 100 characters per line.

Checking and Correcting Coding Style

The coding style can be checked easily by using cpplint.
The warning criteria have been slightly relaxed, as follows.
  • Specify --linelength=100 to allow up to 100 characters per line.
  • Warnings due to cpplint's legal/copyright, build/include_alpha, and whitespace/newline categories have been suppressed.
  • Includes of the algorithm header are not required when using std::swap.
You can run checks easily by writing rules like the following in your makefile.
cpplint:
-./cpplint.py --linelength=100 \
$(shell find ./include/ -path "*.h") \
$(shell find ./sources/ -path "*.cpp" -or -path "*.h") 2>&1 | tee cpplint.log
In addition, nlib supports checking source code using clang-check. Generating a makefile with CMake creates a file called compile_commands.json, which clang-check uses to run.
For more information, see JSON Compilation Database Format Specification in the clang manual.

Automatic Formatting of Coding Style

You can also use clang-format to standardize the indentation and other stylistic aspects to suit your preferences. You can use the configuration used by nlib, which is stored in a .clang-format file (for clang 3.5) that is included in the nlib package. It is recommended that you first convert your source code en masse using clang-format, and then use cpplint.py to do common detailed checks.

Setting Warnings During Compilation

Code quality can be improved by testing with multiple compilers and runtime environments. Configure all compilers to relatively strict warning levels.
  • Ensure successful compilation with Visual Studio 2013 and all newer versions. In general, resolve all warnings shown when using the /W4 option.
    • Suppression of pragma warnings is permitted.
    • Make sure that the latest updates are always applied, and check _MSC_FULL_VER in your code.
    • Periodically reduce warnings using code analysis in Visual Studio.
  • Make sure that compilation succeeds using the latest versions of gcc and clang binary packages available for Cygwin (32-bit) and Ubuntu. In general, resolve all warnings shown when using -Wall -Wextra.
    • Always use the newest version of Cygwin. If a Cygwin update causes a build not to work as expected, revise the build scripts.
    • Suppression of pragma warnings is permitted.
    • Use -Wstack-usage=65536 to ensure that the stack is not overused.
    • Use -fstack-protector.
  • Make sure that compilation succeeds without warnings in the CAFE development environment.
  • Make sure that compilation succeeds in the CTR development environment.
  • It is preferable to have compilation succeed in all 64-bit environments.
  • Also resolve all Doxygen warnings.
The basic release conditions for nlib are ensuring that there are zero cpplint.py warnings and that compilation and tests succeed in all environments.

Additional Rules and Conventions

  • In general, code for functions that are not declared or implemented in functions defined in Platform.h must appear within code that can be compiled in all environments it could conceivably be used in. That said, inclusion of code that is optimized for specific hardware using SIMD instructions or other such techniques is permitted.
  • Use macros like NLIB_FINAL or NLIB_OVERRIDE and typedefs to safely incorporate C++11 features and continuously make it possible to port to C++11 or C++14.
  • Use POSIX.1 2008 error codes for error values whenever possible. Use the errno_t type, as recommended by https://www.securecoding.cert.org/confluence/x/QgAy.
    For example:
    typedef int errno_t;
    Using code like this makes it easier to understand that these are error values.
    • Values of POSIX.1 2008 error codes are sufficiently small integer values, so immediate values can be loaded efficiently on processors like ARM.
  • They must be implemented without using exceptions or RTTI. Make sure that your application code (that calls nlib) is written in a way that anticipates the use of these features, such as exception safety.
  • Instead of functions like malloc and free, use functions like nlib_malloc and nlib_free.
  • Use new (std::nothrow) for new.
  • The global new operator must not be overloaded within the library.
    • This overloading is not allowed because it affects the behavior of other libraries that you wouldn't think of as having dependencies.
  • Encode multi-byte strings in either ASCII or UTF-8.
    • The implementation must allow UTF-8, including in Windows. Path strings and strings passed to nlib_printf are passed to Win32 API after conversion to UTF-16 in nlib.
    • Command-line argument processing (Nflags) is an exception.
  • The char type must work correctly, whether signed or unsigned.
    • These can be handled explicitly as signed values by using signed char.
  • UTF-16 is supported if wchar_t is 16-bit; UTF-32 is supported if wchar_t is 32-bit.
    • NLIB_WCHAR_SIZE is defined as either 2 or 4, so you can use this to make the determination.
  • Use nlib_utf16_t as the type for UTF-16 characters, and use nlib_utf32_t as the type for UTF-32 characters.
    • In C++11, these types are defined with typedef as char16_t and char32_t, respectively.
  • In general, prioritize UTF-8 over UTF-16 and UTF-32.
    • UTF-8 consumes less memory, and is easy to support on multiple platforms.
    • The existence of surrogate pairs and combining characters make it impossible to always represent a single character with a fixed length, even if using UTF-16 or UTF-32.
    • Using nlib_strcplen_ex allows you to request the number of code points and byte length simultaneously while performing validation. If SIMD is supported, this operation can be sped up even more.
  • Move constructors and move assignment operators must be defined in code separated by NLIB_CXX11_RVALUE_REFERENCES, and you must also define member functions like the following so that the same functionality can be implemented in C++03.
    Type::Type(Type& rhs, ::nn::nlib::move_tag); // Behaves like the move constructor.
    Type& Type::assign(Type& rhs, ::nn::nlib::move_tag); // Behaves like the move assignment operator.
    Excellent alternative approaches like boost::move exist, but they cannot achieve full compatibility, so they are not used.
  • type_traits are used with care. This feature of C++11 is sometimes buggy or is not a complete implementation for some compilers or compiler versions. A tentative wrapper has been provided in nn/nlib/TypeTraits.h, but only use it if there is no other alternative.
  • Do not use non-copyable mixins.
    • Use NLIB_DISALLOW_COPY_AND_ASSIGN instead.
    • The increase in levels of inheritance can sometimes have a negative effect on source code analysis programs and documentation systems.
  • For the C standard library headers, include files in .h format. There are two reasons for this.
    • It handles situations when the C++ standard library headers are corrupted.
    • It minimizes code confusion caused by the inclusion of both %<xxx.h%> and %<cxxx%> formats within nlib. (The %<xxx.h%> format must be included because there is a requirement that nn/nlib/Platform.h can be compiled with a C compiler.)
  • Write tests in Google Test format, and make sure unit tests have been run on all platforms.
    • This makes it easier to find bugs that only occur in certain environments, in addition to platform-specific implementation bugs.
    • It is best if you separate the common tests to be run on any platform from the PC-specific tests.
      • This makes it easier to write stress tests and brute-force tests in environments with as much power as a PC.
      • On a PC environment, it is sometimes easy and convenient to use open-source software (OSS) in test programs.
      • In a Windows environment, DirectX and other such APIs can be used in test programs, which is sometimes convenient.
      • In environments where Google Test does not run, the testing library can be used.
      • When compiling using gcc, you can calculate test coverage automatically by using gcovr.
  • Write the Reference Manual in Doxygen format.
    • Doxygen is continuously being improved, and it is a documentation system that we can expect to exist and be improved even five years from now.
    • Doxygen comments are written within manual.h files located within each include directory. English versions of the manual are stored as manual_en.h. This approach allows multiple languages to be managed within the same repository.
  • If possible, enable building using CMake.
    • CMake is a representative selection for cases when you are coding in C/C++ that must run on both Windows and Linux.

Support for CMake in nlib

CMake is used to compile nlib samples and nlib itself, and can be used by nlib users. CMake is used by including CMake support scripts, such as cmake/common.cmake and cmake/import.nx_misc.cmake, from CMakeLists.txt. You can refer to samples/cmake/CMakeLists.txt.
CMake support scripts provided by nlib have been verified for their operations with the following compilers.
  • Visual Studio of any of its versions
  • gcc used with Cygwin
  • gcc and clang used with Bash on Ubuntu running on Windows
  • gcc and clang running on Linux
  • clang running on FreeBSD
  • clang running on macOS
  • ghs (cmake/toolchain-cafe-ninja.cmake is used) running on CAFE. CMake and ninja (https://ninja-build.org/) for Windows are required.
  • armcc used with CTR (using cmake/toolchain-ctr-ninja.cmake). CMake and ninja (https://ninja-build.org/) for Windows are required.

FAQ

This section explains commonly encountered problems while using nlib, and their workarounds.
Overloading new/delete
nlib uses new (std::nothrow) for memory allocation. When applications and other libraries overload new or delete, operator new(size_t, const std::nothrow_t&) must also be overloaded.
For more information, see the heap_replace_malloc sample.
Overriding Functions Like malloc
nlib does not use malloc or free directly. Instead, it uses functions like nlib_malloc and nlib_free. These functions are weakly defined, so they can be overridden by user-defined functions.
You must override these functions if you want to use a user-defined heap. The following functions are required.
void* nlib_malloc(size_t size);
void nlib_free(void* ptr);
void* nlib_calloc(size_t nmemb, size_t size);
void* nlib_realloc(void* ptr, size_t size);
void* nlib_memalign(size_t alignment, size_t size); // Not defined if WIN32 or CTR.
For more information, see the heap_replace_malloc sample.
Compile Options
Platform-specific switches and other such definitions are defined in files such as cmake/common.cmake.win32 within the package. Please reference them.
Compilation results in "MACRO_XXXX not defined, compile may fail" being displayed.
You need to define MACRO_XXXX using the command-line arguments to the compiler.
Should we use the latest version for our project?
That's best left up to the judgment of the project using nlib. Writing acceptance tests for future use makes it easier to make this type of decision. However, only the latest version receives support. Note this fact.
Please release a special version of the package for our project.
There are as many project requirements as there are projects. For example, in the past, inquiries like the following have been received.
  • We don't need library X, so we'd like to have a special release with that removed.
  • We'd like a release without any samples.
  • We are using the 64-bit version of Visual Studio 2015, so we'd like a release that comes with a pre-built Visual Studio 2015 solution.
To handle such requirements in a flexible manner, the projects using nlib are permitted to restructure or repackage nlib and to redistribute it, but only within the project.
Why can't text encoded using Shift-JIS be displayed with nlib_printf?
The input string for nlib_printf must be encoded using UTF-8.
There is a bug in a version released half a year ago. Will you please release a patch that fixes it?
Bug fixes are applied to the latest version. You need to use a newer release that incorporates the fix.
Will you please make it follow our library or framework's approach?
If your approach is publicly available, and is supported by all platforms (Windows, Linux, FreeBSD, OS X, Cafe, and CTR), this might be possible. Please get in touch.

Revision History

2017-05-16

  • NLIB-27: Added the nlib_byte_t type. In the future, this type will be defined as std::byte using a typedef in C++17.
  • NLIB-41: All enumerator values will be written in the format of kEnumName in the future. Enumerator values in the format of ENUM_NAME will no longer be allowed to be used.
  • NLIB-48: Deleted the nlib_strcplen2, nlib_popcnt(x), nlib_clz(x), and nlib_ctz(x) macros that had been defined only for compatibility. Use nlib_strcplen, nlib_popcnt32(x), nlib_clz32(x), and nlib_ctz32(x) instead.
  • msgpack::MpObject::Clone() was changed to a const member function.
  • Made other minor changes and fixes.

2017-04-18

  • Fixed a bug where msgpack::MpWalker generated an error status when the end of a byte sequence was reached.
  • Made other minor changes and fixes.

2017-03-22

  • The compatibility of the source code is partially lacked in this release.
  • Added the msgpack::JsonPatch class and the msgpack::JsonPatchBuilder class. Those classes support JSON Patch(RFC 6902).
  • Fixed an issue where the definition of simd::i128arg unnecessarily represents a reference type.
  • To msgpack::GetMapItem(), added an overload that can obtain MpObjectKv using a non-null terminated string.
  • Deprecated codes have been deleted.
  • NLIB-17: The /Ob1 option is now specified for a Debug build in Visual Studio. This change causes inline functions to be expanded.
  • Made other minor changes and fixes.

2017-02-14

2017-01-24

  • Added msgpack::MpObject::ResolveJsonPointer() to the msgpack library. This class supports JSON Pointer (RFC6901).
  • nlib_utf16_t and nlib_utf32_t can now be defined with typedef even when using C.
  • Newly added the nlib_utf8_t type.
  • utf16_t and utf32_t are now deprecated. These types will be deleted in a future release.
  • NINTENDOSDK-3321: Fixed an issue where msgpack::jsonrpc::JsonRpcRequest::Init() determined a request to be invalid that no params is specified for.
  • Made other minor changes and fixes.
Contains Past Revision Histories.