nlib
nlib Main Page

This manual is a collection of libraries that support Windows, Linux, FreeBSD, OS X, Cygwin, and Nintendo's development environments.

About nlib

nlib is being developed to provide a group of user-space libraries that enable you to program for Windows, Linux, FreeBSD OS X, Cygwin, and Nintendo's development environments using the same interface. Its development follows a software-centric approach, adopts appropriate and pragmatic coding rules (nlib Coding Rules), and implements an interface that is independent from the individual development environments. This focus makes it possible to develop in parallel for multiple development environments while maintaining high quality, performance, and reliability.

Features Implemented by nlib

The main features implemented are listed below.
  • Reading and writing XML and JSON
  • Highly-efficient memory allocators
  • Test framework compatible with Google Test
  • Wrapper libraries that allows codes to be developed that supports both SSE4.2 and NEON. Includes support for various calculations for 3x4 matrices, and three- and four-dimensional vectors.
  • Succinct data structure libraries for features like keyword searching
  • UTF-8 reading, writing, normalization, and conversion from and to UTF-16 and UTF-32
  • Threads, mutexes, condition variables, TLS, barriers, read-write locks, and atomic variables
  • Getting and freeing system memory, getting the time, and reading and writing files
  • Basic lock-free data structure and algorithms
  • Functions and macro definitions for base C linkages used to provide multi-platform and multi-compiler support

nlib Operating Environment

Currently, nlib has been verified for compilation and operation in the following environments.
Platform Library types Remarks
Windows Visual Studio 201764bitstatic library, DLLVersion 15.7.27703.2026
Visual Studio 2015 64bit static library, DLL Update 3
32bit static library
Visual Studio 2013 64bit static library, DLL Update 5, Express Edition is available
32bit static library
Cygwin64bitstatic librarygcc 7.3.0, newlib, libstdc++
Windows Subsystem for Linux64bitshared libraryUbuntu 16.04, gcc 5.4.0, clang 3.8.0
Linux Ubuntu 18.04 64bit shared library gcc 7.3.0, clang 6.0.0, glibc, libstdc++, ubuntu/bionic64 (virtualbox, 20180531.0.0)
Ubuntu 16.04 LTS 64bit gcc 5.3.1, clang 3.8, glibc, libstdc++, bento/ubuntu-16.04 (virtualbox, 201803.24.0)
CentOS 7 64bit devtoolset-4, gcc 5.3.1, glibc, libstdc++, centos/7 (virtualbox, 1804.02)
Fedora 28 64bit gcc 8.0.1, clang 6.0.0, glibc, libstdc++. fedora/28-cloud-base (virtualbox, 20180425)
Alpine 3.6.2 64bit gcc 6.3.0, musl libc, libstdc++, maier/alpine-3.6-x86_64 (virtualbox, 3.6.2)
FreeBSD FreeBSD 11.1 64bit shared library clang 4.0, BSD libc, libc++, freebsd/FreeBSD-11.1-STABLE (virtualbox, 2018.04.12)
macOS macOS 10.13 (High Sierra) 64bit shared library Xcode 9.4
CTR CTR-SDK 11.5 32bit static library CTR_SDK-11_5_2-20161105-ja.zip
armcc_4_1_nintendo-b1454-20150227.zip
Cafe CafeSDK 2.13.01 32bit static library cafe_sdk-2_13_01-20150728.zip
MULTI-5_3_27-20170829.zip
Depending on the compiler version and other factors, compilation uses C++11 features as appropriate (C++11/14/17 Features Used by nlib).

nlib Support Policy

Please contact the site where the package was released for instructions about how to get support. Also read the documentation, including the FAQ. This document and the code samples explain the current implementation. Future development plans and release dates will not be documented or made public. Support and bug fixes will be made in the latest version at that time. Special versions will not be provided for specific projects, and no special support will be provided. This rule is essential to keep the size of the development team in check.
Unannounced changes that break compatibility may occur if there is no way around it. Note this fact.
Before use, it is probably a good idea to perform some acceptance tests to determine performance criteria including whether the package works as expected, is sufficiently fast, and has memory consumption within the acceptable limits. Creating acceptance tests up front also helps to avoid trouble when upgrading nlib. It is a good idea to use the testing library to create these types of acceptance tests.
The bug may be quickly fixed if you describe the code reproducing the bug as a test code in the nn::nlib::testing format and include it in your bug report.

nlib Package Description

Note that the included libraries and supported platforms vary depending on the package.

Directory Structure

The package is structured as follows.
    ├── bin
    │   └── cmake                 # Command-line tools
    ├── cmake                      # Common Cmake scripts
    ├── documents-ja(documents-en)
    │   └── API                   # Reference Manual
    │       ├── html              # HTML version of the Reference Manual
    │       └── searchdata.xml    # Search index
    ├── include                    # Include files (in `system/include` for Cafe)
    │   ├── nn
    │   │   └── nlib
    │   │       ├── exi
    │   │       ├── heap
    │   │       ├── msgpack
    │   │       │   └── jsonrpc
    │   │       ├── oss
    │   │       ├── simd
    │   │       ├── succinct
    │   │       ├── testing
    │   │       ├── threading
    │   │       └── unicode
    │   ├── lz4.h
    │   ├── lz4hc.h
    │   ├── lz4frame.h
    │   ├── sqlite3.h
    │   └── sqlite3ext.h
    ├── lib                        # Platform-specific library files (`system/lib` for Cafe, `libraries` for CTR)
    ├── samples                    # Source code for samples
    │  ├── cmake
    │  └── sources
    │      ├── exi
    │      ├── heap
    │      ├── misc
    │      ├── msgpack
    │      ├── oss
    │      ├── succinct
    │      └── testing
    └── setup                      # Development environment setup scripts

Platform-Specific Descriptions

Descriptions of Each Library

libn Library (including libn.a, libn.so, n-vc120amd64.lib, and n-vc120dllamd64.dll, etc.)
This library includes functions that have C linkages for suppressing differences among various platforms.
Specifically, the following functions are provided:
  • Functions, including nlib_epochtime(), that obtain the current time.
  • The nlib_gen_random() function that obtains random values.
  • Allocation and deallocation of (virtual) memory from the operating system
  • Functions and macros supporting pthread
  • Message queues supporting lock-free behavior (nlib_mq)
  • Functions, including nlib_atomic_compare_exchange32(), that atomically rewrite memory.
  • Functions, including nlib_debug_break() and nlib_debug_backtrace(), that access the debugger.
  • The nlib_getenv() function that obtains environment variable values.
  • File handles and file I/O functionality, including nlib_fd_open().
  • Functions relating to string searches, accelerated with SIMD.
  • String encoding conversion between UTF-8 and UTF-16 and between UTF-8 and UTF-32, in both directions
  • Functions, including nlib_strto_int32(), that wrap standard C functions, including strtol() and strtoll(), in a way they are not wrongly used.
  • Functions, including nlib_isspace(), that implement standard C functions defined by ctype.h in a way they are not affected by the locale.
  • Functions, including nlib_snprintf(), that are adjusted to suppress differences among the environment-specific behaviors of snprintf(), snwprintf(), and other similar functions to prevent them from being wrongly used.
  • Fast conversion of endianness (including nlib_swapendian_32())
  • Fast CRC32 and CRC32C calculations (nlib_crc32() and nlib_crc32c())
  • Fast pop count calculations (including nlib_popcnt64())
  • Macros that absorb differences among the various attributes for each compiler
misc Library (libnx_misc.a, libnx_misc.so, nx_misc-vc120amd64.lib, nx_misc-vc120dllamd64.dll, etc.)
This library is a class library that includes relatively small classes. It implements many features that are believed to be commonly used in programs.
Specifically, it includes the following. The namespaces include nn::nlib, nn::nlib::unicode, and nn::nlib::threading.
exi Library (libnx_exi.a, libnx_exi.so, nx_exi-vc120amd64.lib, nx_exi-vc120dllamd64.dll, etc.)
A library used to read and write binary XML. It can also read and write text-based XML. It supports XML namespaces and includes a simple, easily programmable pull parsing API.
The namespace is nn::nlib::exi.
heap Library (libnx_heap.a, libnx_heap.so, nx_heap.lib, nx_heap-vc120dllamd64.dll, etc.)
This library is a heap library for which memory allocation and deallocation is optimized with multithreaded operations. The nmalloc and nfree functions (and the heap::CachedHeap and heap::CentralHeap classes used to implement them) keep performance degradation in check by controlling fragmentation through the use of multi-level free lists. By maintaining per-thread caches for small blocks of memory, memory can be allocated and freed without locking, which helps to limit the degradation of multithreaded performance. Pointers within the free lists are checked for both mangling and their allocation status. This implementation makes it more difficult to use them in an invalid way by overwriting memory externally.
The namespaces are nn::nlib::heap and the global namespace (C linkage functions).
succinct Library (libnx_succinct.a, libnx_succinct.so, nx_succinct-vc120amd64.lib, nx_succinct-vc120dllamd64.dll, etc.)
A library used to handle succinct data structures. You can think of a succinct data structure as one whose content can be searched while it is still compressed.
The succinct library includes the following.
  • Implementation of succinct vector classes
  • A class that implements balanced parentheses
  • A class that implements LOUDS (level-order unary degree sequences)
  • Trie implementation
  • A class for creating high-compression indices and searching strings using the Aho–Corasick algorithm
The namespace is nn::nlib::succinct.
msgpack Library (libnx_msgpack.a, libnx_msgpack.so, nx_msgpack-vc120amd64.lib, nx_msgpack-vc120dllamd64.dll, etc.)
A library for reading and writing MessagePack and JSON. It also includes a class for parsing CSV.
The msgpack library includes the following.
  • Classes for reading and writing JSON from streams
  • Classes for reading and writing MessagePack from streams
  • A class for high-speed in-memory MessagePack parsing
  • A class for reading CSV
  • Classes for storing JSON, MessagePack, and CSV data after it has been read
  • Implementation of JSON-RPC
The namespace is nn::nlib::msgpack.
testing Library (libnx_testing.a, libnx_testing.so, nx_testing-vc120amd64.lib, nx_testing-vc120dllamd64.dll, etc.)
A library for writing highly portable test code. It is a testing framework that is somewhat source-code compatible with tests written for Google Test.
The namespace is nn::nlib::testing. (It is aliased to the testing namespace for the sake of Google Test compatibility.) It depends on the misc library.
oss Library (libnx_oss_bsdiff.a, libnx_oss_sqlite3.a, libnx_oss_lz4.a, etc)
Contains code for open-source software (OSS), code derived from OSS, and code for accessing OSS. To use it, an OSS license must be processed.
The following libraries are currently included. They are built as separate library files.
  • A library conversion of bsdiff that has been modified to use zlib instead of bzlib2, and the corresponding command-line programs
  • A version of SQLite3 that has been modified to run using the interfaces provided by nlib, and a corresponding command-line shell
  • LZ4 is approximately 20 times faster at data compression than zlib, and approximately five times faster at decompression.
The namespaces are nn::nlib::oss and the global namespace (C linkage functions).
simd Library (Mostly Inline Functions Included in the misc Library)
SIMD instructions can significantly increase the speed of programs. The simd library makes it possible to write code that supports both SSE4.2 and NEON.
It supports integer and single-precision floating-point number calculations that enable your code to compute at high speeds.
Single-precision floating-point number calculation can be used even in environments that do not support SSE or NEON.
On Cafe, this is optimized to use paired singles.
In other words, the same code can support the following environments.
  • On x86 processors, the code is compiled with speed boosts through the use of SSE 4.2.
  • On processors with NEON support, the code is compiled with speed boosts through the use of NEON.
  • On Cafe, the code is compiled with speed boosts through the use of paired singles. (Floating-point operations alone are quite a bit slower than the Cafe-SDK for operations like matrix calculations, because the interface returns values for data like four-dimensional vectors.)
  • On CTR, generic code is compiled (floating-point operations only).
The namespace is nn::nlib::simd.

Dependency between nlib libraries.

The dependency between nlib libraries is as shown in the following figure.

dot_inline_dotgraph_34.png

C++11/14/17 Features Used by nlib

nlib is compiled using C++11/14/17 features for some combinations of compilers and versions. Whether the compiler supports those features is determined based on the compiler version and using Feature Testing Macros recommended in C++14 and later.
C++11 Feature Identification Macro Macros and Typedefs for C++03 Compatibility
Rvalue references __cpp_rvalue_references None 2010 4.3
Variadic templates __cpp_variadic_templates None 2013 4.7
Static assertions __cpp_static_assert NLIB_STATIC_ASSERT() 2010 4.3
Null pointer constant NLIB_CXX11_NULL_POINTER_CONSTANT nn::nlib::nullptr, nn::nlib::nullptr_t 2010 4.6
Deleted functions NLIB_CXX11_DEFAULTED_AND_DELETED_FUNCTIONS NLIB_DISALLOW_COPY_AND_ASSIGN() 2015 4.4
Explicit conversion operators NLIB_CXX11_EXPLICIT_CONVERSION_OPERATORS NLIB_SAFE_BOOL() 2013 4.6
Alias templates __cpp_alias_templates None 2013 4.7
constexpr __cpp_constexpr NLIB_CEXPR 2015 4.7
char16_t, char32_t __cpp_unicode_characters nlib_utf16_t, nlib_utf32_t 2015 4.4
alignas NLIB_CXX11_ALIGNMENT_SUPPORT NLIB_ALIGNAS() 2015 4.8
Explicit virtual overrides NLIB_CXX11_EXPLICIT_VIRTUAL_OVERRIDES NLIB_OVERRIDE 2012 4.7
final NLIB_CXX11_EXPLICIT_VIRTUAL_OVERRIDES NLIB_FINAL 2012 4.7
noexcept NLIB_CXX11_NOEXCEPT NLIB_NOEXCEPT 2015 4.6
Range-based for __cpp_range_based_for None 2012 4.6
lambda __cpp_lambdas None 2012 4.7
Delegating constructors __cpp_delegating_constructors None 2013 4.7
std::unique_ptr NLIB_CXX11_UNIQUEPTR nn::nlib::UniquePtr 2013 4.7
#include <random> NLIB_CXX11_STDLIB_RANDOM None 2013 4.8
#include <chrono> NLIB_CXX11_STDLIB_CHRONO Conversion to and from nn::nlib::DateTime 2012 4.7
#include <atomic> NLIB_CXX11_STDLIB_ATOMIC NLIB_MEMORY_ORDER_SEQ_CST and others 2012 4.8

nlib Coding Rules

nlib Basic Coding Rules

The nlib coding rules are a slight modification and relaxation of the latest version of the Google C++ Style Guide (and a slightly outdated Japanese translation).
The reasons for using the Google C++ Style Guide are listed below.
  • It is easy to access documentation about the coding rules. Translations are available (albeit unofficial ones).
  • Similar to the general coding rules for the gaming industry, C++ exceptions and RTTI use are prohibited.
  • It has been updated continuously in the past.
  • Changes made to the coding rules with each update are reasonable, and there haven't been any sudden, drastic changes.
  • We can expect it to be maintained even five years from now.
The changes and relaxations adopted for nlib are shown below.
Coding rule Google C++ Style Guide nlib
Line length 80 100
Indentation 2 4
Parameter order Inputs followed by outputs Outputs followed by inputs
Default argument Prohibited in principle Allowed for use except for the cases described below:
  • In the case of a constructor. Issues may arise if the default value allows an argument not to be explicitly specified until only one argument remains unhandled.
  • In the case of a virtual function. Issues may arise if the overridden function has a default argument different from that of the original function.
  • In the case where the argument can be used as a function pointer. Confusion may be induced in the coding.
File name file_name.cc FileName.cpp(file_name.cpp in the sample)
#include guard Also append #pragma once (for accelerating compilation in Visual Studio)
Use of boost Not to be used by libraries (may be used for other purposes such as testing)
Accessor function name get_xyz() / set_xyz() GetXyz() / SetXyz()
Argument passed by reference Non-const references are prohibited In nlib, non-cost references are allowed as exceptions in the following cases.
  • When specializing std::swap
  • Code that imitates move semantics in C++03
  • Class with simple features. For example, a class having a feature that only maintains a lock, such as the ScopedLock class.
  • When passing nn::nlib::UniquePtr as an argument (Returning a value would be inefficient, and it would be strange to pass a pointer to UniquePtr.)
Use of C++11 Ensure that headers and source code can be properly compiled also by using C++03 compilers.
  • This doesn't mean you can't use C++11. Features such as move constructors, range-based for statements, explicit virtual overrides, and explicit conversion operators should all be supported proactively.
  • If possible, the same source code is compiled using macros and other preprocessor directives. If not possible, implement similar features that can be used from C++03.
nullptr Use if C++11 In principle, don't use nullptr_t or nullptr as long as C++03 is used.
  • The nullptr idiom for implementing nullptr in C++03 results in errors if it is defined in multiple libraries, so it cannot be defined in the global namespace.
  • With nlib, it is defined as nn::nlib::nullptr_t and nn::nlib::nullptr for C++03.
  • Avoiding the use of misleading overloads is a workaround for most cases that would require nullptr.
constexpr Use if C++11 For the time being, use of constexpr should be avoided except in obvious cases.
  • Unfortunately, code that uses constexpr proactively has a drawback in terms of portability, because differences among compiler behavior can result in many compilation errors.
  • "Obvious cases" refers to null constant objects or cases that one would expect to compile without issues on any compatible compiler that supports constexpr (for example, simple constructors that merely set constants).
Comment
  • Considering translation costs, do not use non-ASCII (non-English) characters in comments for public headers (This rule does not apply if the content should be never translated).
  • If you want to write detailed comments, include them in either source code files or the Reference Manual.
  • Doxygen comments must not be written in public headers. This causes recompilation due to changes in the documentation, and increases the translation burden. Files such as manual_ja.h and manual_en.h should be prepared for each library and comments should be written into those files.
  • In the reference manuals, never place newline characters in the middle of a sentence. This rule is meant to support the nature of the translation-aid software used for localization. It takes precedence over the previously mentioned limit of 100 characters per line.

Checking and Correcting Coding Style

The coding style follows clang-format as its rule. .clang-format is held in the root directory and updated as necessary.

Setting Warnings During Compilation

Code quality can be improved by testing with multiple compilers and runtime environments. Configure all compilers to relatively strict warning levels.
  • Ensure successful compilation with Visual Studio 2013 and all newer versions. In general, resolve all warnings shown when using the /W4 option.
    • Suppression of pragma warnings is permitted.
    • Make sure that the latest updates are always applied, and check _MSC_FULL_VER in your code.
    • Periodically reduce warnings using code analysis in Visual Studio.
  • Make sure that compilation succeeds using the latest versions of gcc and clang binary packages available for Cygwin (32-bit) and Ubuntu. In general, resolve all warnings shown when using -Wall -Wextra.
    • Always use the newest version of Cygwin. If a Cygwin update causes a build not to work as expected, revise the build scripts.
    • Suppression of pragma warnings is permitted.
    • Use -Wstack-usage=65536 to ensure that the stack is not overused.
    • Use -fstack-protector.
  • Make sure that compilation succeeds without warnings in the CAFE development environment.
  • Make sure that compilation succeeds in the CTR development environment.
  • It is preferable to have compilation succeed in all 64-bit environments.
  • Also resolve all Doxygen warnings.

Additional Rules and Conventions

  • In general, code for functions that are not declared or implemented in functions defined in Platform.h must appear within code that can be compiled in all environments it could conceivably be used in. That said, inclusion of code that is optimized for specific hardware using SIMD instructions or other such techniques is permitted.
  • Use macros like NLIB_FINAL or NLIB_OVERRIDE and typedefs to safely incorporate C++11 features and continuously make it possible to port to C++11 or C++14.
  • Use POSIX.1 2008 error codes for error values whenever possible. Use the errno_t type, as recommended by https://www.securecoding.cert.org/confluence/x/QgAy.
    For example:
    typedef int errno_t;
    Using code like this makes it easier to understand that these are error values.
    • Values of POSIX.1 2008 error codes are sufficiently small integer values, so immediate values can be loaded efficiently on processors like ARM.
  • They must be implemented without using exceptions or RTTI. Make sure that your application code (that calls nlib) is written in a way that anticipates the use of these features, such as exception safety.
  • Instead of functions like malloc and free, use functions like nlib_malloc and nlib_free.
  • Use new (std::nothrow) for new.
  • The global new operator must not be overloaded within the library.
    • This overloading is not allowed because it affects the behavior of other libraries that you wouldn't think of as having dependencies.
  • Encode multi-byte strings in either ASCII or UTF-8.
    • The implementation must allow UTF-8, including in Windows. Path strings and strings passed to nlib_printf are passed to Win32 API after conversion to UTF-16 in nlib.
    • Command-line argument processing (Nflags) is an exception.
  • The char type must work correctly, whether signed or unsigned.
    • These can be handled explicitly as signed values by using signed char.
  • UTF-16 is supported if wchar_t is 16-bit; UTF-32 is supported if wchar_t is 32-bit.
    • NLIB_WCHAR_SIZE is defined as either 2 or 4, so you can use this to make the determination.
  • Use nlib_utf16_t as the type for UTF-16 characters, and use nlib_utf32_t as the type for UTF-32 characters.
    • In C++11, these types are defined with typedef as char16_t and char32_t, respectively.
  • In general, prioritize UTF-8 over UTF-16 and UTF-32.
    • UTF-8 consumes less memory, and is easy to support on multiple platforms.
    • The existence of surrogate pairs and combining characters make it impossible to always represent a single character with a fixed length, even if using UTF-16 or UTF-32.
    • Using nlib_strcplen_ex allows you to request the number of code points and byte length simultaneously while performing validation. If SIMD is supported, this operation can be sped up even more.
  • Move constructors and move assignment operators must be defined in code separated by __cpp_rvalue_references, and you must also define member functions like the following so that the same functionality can be implemented in C++03.
    Type::Type(Type& rhs, ::nn::nlib::move_tag); // Behaves like the move constructor.
    Type& Type::assign(Type& rhs, ::nn::nlib::move_tag); // Behaves like the move assignment operator.
    Excellent alternative approaches like boost::move exist, but they cannot achieve full compatibility, so they are not used.
  • type_traits are used with care. This feature of C++11 is sometimes buggy or is not a complete implementation for some compilers or compiler versions. A tentative wrapper has been provided in nn/nlib/TypeTraits.h, but only use it if there is no other alternative.
  • Do not use non-copyable mixins.
    • Use NLIB_DISALLOW_COPY_AND_ASSIGN instead.
    • The increase in levels of inheritance can sometimes have a negative effect on source code analysis programs and documentation systems.
  • For the C standard library headers, include files in .h format. There are two reasons for this.
    • It handles situations when the C++ standard library headers are corrupted.
    • It minimizes code confusion caused by the inclusion of both %<xxx.h%> and %<cxxx%> formats within nlib. (The %<xxx.h%> format must be included because there is a requirement that nn/nlib/Platform.h can be compiled with a C compiler.)
  • Write tests in Google Test format, and make sure unit tests have been run on all platforms.
    • This makes it easier to find bugs that only occur in certain environments, in addition to platform-specific implementation bugs.
    • It is best if you separate the common tests to be run on any platform from the PC-specific tests.
      • This makes it easier to write stress tests and brute-force tests in environments with as much power as a PC.
      • On a PC environment, it is sometimes easy and convenient to use open-source software (OSS) in test programs.
      • In a Windows environment, DirectX and other such APIs can be used in test programs, which is sometimes convenient.
      • In environments where Google Test does not run, the testing library can be used.
      • When compiling using gcc, you can calculate test coverage automatically by using gcovr.
  • Write the Reference Manual in Doxygen format.
    • Doxygen is continuously being improved, and it is a documentation system that we can expect to exist and be improved even five years from now.
    • Doxygen comments are written within manual.h files located within each include directory. English versions of the manual are stored as manual_en.h. This approach allows multiple languages to be managed within the same repository.
  • If possible, enable building using CMake.
    • CMake is a representative selection for cases when you are coding in C/C++ that must run on both Windows and Linux.

Support for CMake in nlib

CMake is used to compile nlib samples and nlib itself, and can be used by nlib users. Include cmake/common.cmake from CMakeLists.txt and call libraries within nlib to be used with FIND_PACKAGE(). You can refer to samples/cmake/CMakeLists.txt.
CMake support scripts provided by nlib have been verified for their operations with the following compilers.
  • Visual Studio of any of its versions
  • gcc used with Cygwin
  • gcc and clang used with Windows Subsystem for Linux(Ubuntu)
  • gcc and clang running on Linux
  • clang running on FreeBSD
  • clang running on macOS
  • ghs (cmake/toolchain-cafe-ninja.cmake is used) running on CAFE. CMake and ninja (https://ninja-build.org/) for Windows are required.
  • armcc used with CTR (using cmake/toolchain-ctr-ninja.cmake). CMake and ninja (https://ninja-build.org/) for Windows are required.

FAQ

This section explains commonly encountered problems while using nlib, and their workarounds.
Overloading new/delete
nlib uses new (std::nothrow) for memory allocation. When applications and other libraries overload new or delete, operator new(size_t, const std::nothrow_t&) must also be overloaded.
For more information, see the heap_replace_malloc sample.
Overriding Functions Like malloc
nlib does not use malloc or free directly. Instead, it uses functions like nlib_malloc and nlib_free. These functions are weakly defined, so they can be overridden by user-defined functions.
You must override these functions if you want to use a user-defined heap. The following functions are required.
void* nlib_malloc(size_t size);
void nlib_free(void* ptr);
void* nlib_calloc(size_t nmemb, size_t size);
void* nlib_realloc(void* ptr, size_t size);
void* nlib_memalign(size_t alignment, size_t size); // Not defined if WIN32 or CTR.
For more information, see the heap_replace_malloc sample.
Compile Options
Platform-specific switches and other such definitions are defined in files such as cmake/common.cmake.win32 within the package. Please reference them.
Compilation results in "MACRO_XXXX not defined, compile may fail" being displayed.
You need to define MACRO_XXXX using the command-line arguments to the compiler.
Should we use the latest version for our project?
That's best left up to the judgment of the project using nlib. Writing acceptance tests for future use makes it easier to make this type of decision. However, only the latest version receives support. Note this fact.
Please release a special version of the package for our project.
There are as many project requirements as there are projects. For example, in the past, inquiries like the following have been received.
  • We don't need library X, so we'd like to have a special release with that removed.
  • We'd like a release without any samples.
  • We are using the 64-bit version of Visual Studio 2015, so we'd like a release that comes with a pre-built Visual Studio 2015 solution.
To handle such requirements in a flexible manner, the projects using nlib are permitted to restructure or repackage nlib and to redistribute it, but only within the project.
Why can't text encoded using Shift-JIS be displayed with nlib_printf?
The input string for nlib_printf must be encoded using UTF-8.
There is a bug in a version released half a year ago. Will you please release a patch that fixes it?
Bug fixes are applied to the latest version. You need to use a newer release that incorporates the fix.
Will you please make it follow our library or framework's approach?
If your approach is publicly available, and is supported by all platforms (Windows, Linux, FreeBSD, OS X, Cafe, and CTR), this might be possible. Please get in touch.

Revision History

2018-06-05

  • CAFE, CTR, Cygwin, and builds for Visual Studio 2013 will be no longer provided after this version. Note this fact.
  • Updated the version of SQLite3 bundled with the oss libraries to 3.24.0.
  • We have started improving the code quality using libFuzzer(https://llvm.org/docs/LibFuzzer.html).
  • Corrected constness of oss::BsDiffZ() and oss::BsPatchZ() of the oss library.
  • Corrected the issue where oss::BsDiffZ() of the oss library occasionally caused some stacks to be missed due to recursive behaviors during a debug build.
  • Corrected the issue where succinct::AhoCorasickBuilder of the succinct library occasionally made wrong conversion depending on registered words.
  • Corrected the issue that occurred in succinct::SparseSet of the succinct library in a rare case.
  • Corrected the issue that occurred in succinct::Trie::GetCommomPrefixWords() of the succinct library in a rare case.
  • Made modification so that passing a tree with multiple roots to succinct::Bp::Init() of the succinct library causes the conversion to fail.
  • Corrected the issue where executing Import after Export() in some of the classes of the succinct library occasionally caused memory leaks.
  • Corrected the issue where the Windows DLL versions of Uri::operator==() and Uri::operator!=() of the misc library were not exported.
  • Corrected the issue that occurred when an element in Nlist::resize() of the misc library was trivially constructible.
  • Corrected the issue where Uri::Parse() of the misc library occasionally accessed a location with a one-byte offset from the specified range boundary.
  • Other minor changes and fixes

2018-05-08

  • Started operation check in Ubuntu 18.04 LTS. Ubuntu 17.10 is no longer officially supported.
  • Started operation check in Fedora 28. Operation check in Fedora 27 is no longer supported.
  • Added nlib_find_break.
  • Added InputStream::Mark(), InputStream::GoBackToMark(), and InputStream::IsMarkSupported() to InputStream.
  • Updated the version of SQLite3 bundled with the oss libraries to 3.23.1.
  • Updated the version of LZ4 bundled with the oss libraries to 1.8.2.
  • Deleted unicode::UnicodeNormalizer.
  • Deleted unicode::StringPrep.
  • NLIB-161: Fixed an issue where information may be lost from the dump of memory allocation statuses if a memory is failed to be allocated due to an insufficient amount of the physical memory when the virtual memory is enabled.
  • NLIB-164: Fixed an issue where the Windows versions of libraries attempted to reference the debug version of libcurl.
  • NLIB-166: Changed the allocation unit for physical memories from 4 MB to 256 KB, which is applied when using virtual memories in nmalloc. This change provides measures that mitigate an issue that may occur when a small amount of memory is randomly allocated and left not released.
  • Fixed an issue where a memory leak may be found when moving msgpack::MpObject.
  • Made other minor changes and fixes.

2018-04-03

2018-03-13

  • Added nlib_memutf8_to_utf32char().
  • Added simd::I128::SumUint8() and simd::I128::SumUint16().
  • Added an overload that returns the error value together with the result using std::pair and std::tuple.
  • NLIB-145: Fixed an issue where Nlist<int> crashed if it was resized immediately after it was constructed.
  • NLIB-148: Fixed an issue where a correct value was not returned when a double type result was converted with JsonStreamParser::ToFloat().
  • NLIB-150: Fixed an issue where rollback failed after JsonPatch::Apply() failed if a test command is found in Json Patch.
  • Revised the reference manual, and added explanations using the snippet of the code.
  • Made other minor changes and fixes.

2018-01-23

  • Updated the version of SQLite3 bundled with the oss libraries to 3.22.0.
  • Updated the version of LZ4 bundled with the OSS libraries to 1.8.1.2.
  • Added and deleted some samples.
  • Made other minor changes and fixes.
Contains Past Revision Histories.