This manual is a collection of libraries that support Windows, Linux, FreeBSD, OS X, Cygwin, and Nintendo's development environments.
About nlib
nlib
is being developed to provide a group of user-space libraries that enable you to program for Windows, Linux, FreeBSD OS X, Cygwin, and Nintendo's development environments using the same interface. Its development follows a software-centric approach, adopts appropriate and pragmatic coding rules (nlib Coding Rules
), and implements an interface that is independent from the individual development environments. This focus makes it possible to develop in parallel for multiple development environments while maintaining high quality, performance, and reliability.
Features Implemented by nlib
- The main features implemented are listed below.
-
Reading and writing XML and JSON
-
Highly-efficient memory allocators
-
Test framework compatible with Google Test
-
Wrapper libraries that allows codes to be developed that supports both SSE4.2 and NEON. Includes support for various calculations for 3x4 matrices, and three- and four-dimensional vectors.
-
Succinct data structure libraries for features like keyword searching
-
UTF-8 reading, writing, normalization, and conversion from and to UTF-16 and UTF-32
-
Threads, mutexes, condition variables, TLS, barriers, read-write locks, and atomic variables
-
Getting and freeing system memory, getting the time, and reading and writing files
-
Basic lock-free data structure and algorithms
-
Functions and macro definitions for base C linkages used to provide multi-platform and multi-compiler support
nlib Operating Environment
- Currently, nlib has been verified for compilation and operation in the following environments.
Platform | Library types | Remarks |
Windows | Visual Studio 2017 RC | 64bit | static library, DLL | |
Visual Studio 2015 | 64bit | static library, DLL | Update 3 |
32bit | static library |
Visual Studio 2013 | 64bit | static library, DLL | Update 5, Express Edition is available |
32bit | static library |
Cygwin | 64bit | static library | gcc 5.4.0, newlib, libstdc++ |
Bash on Ubuntu on Windows | 64bit | shared library | Ubuntu 14.04 |
Linux | Ubuntu 16.10 | 64bit | shared library | gcc 6.2.0, clang 3.8.1, glibc, libstdc++, https://atlas.hashicorp.com/bento/ |
Ubuntu 16.04 LTS | 64bit | gcc 5.3.1, clang 3.8, glibc, libstdc++, https://atlas.hashicorp.com/bento/ |
Ubuntu 14.04 LTS | 64bit | gcc 4.8.4, clang 3.4, glibc, libstdc++, https://atlas.hashicorp.com/ubuntu/ |
32bit |
CentOS 7 (v1611.01) | 64bit | gcc 4.8.5, glibc, libstdc++, https://atlas.hashicorp.com/centos/ |
Fedora 25 | 64bit | gcc 6.2.1, clang 3.8, glibc, libstdc++. https://atlas.hashicorp.com/fedora/ |
Alpine 3.4.0 | 64bit | gcc 5.3.0, musl libc, libstdc++, https://atlas.hashicorp.com/maier |
FreeBSD | FreeBSD 11.0 | 64bit | shared library | clang 3.8, BSD libc, libc++, https://atlas.hashicorp.com/freebsd |
OS X | OS X 10.11 (El Capitan) | 64bit | shared library | Xcode 8.1 |
CTR | CTR-SDK 11.5 | 32bit | static library | CTR_SDK-11_5_2-20161105-ja.zip
armcc_4_1_nintendo-b1454-20150227.zip |
Cafe | CafeSDK 2.13.01 | 32bit | static library | cafe_sdk-2_13_01-20150728.zip
MULTI-5_3_26-20150918.zip |
- Depending on the compiler version and other factors, compilation uses C++11 features as appropriate (C++11 Features Used by nlib).
nlib Support Policy
- Please contact the site where the package was released for instructions about how to get support. Also read the documentation, including the FAQ. This document and the code samples explain the current implementation. Future development plans and release dates will not be documented or made public. Support and bug fixes will be made in the latest version at that time. Special versions will not be provided for specific projects, and no special support will be provided. This rule is essential to keep the size of the development team in check.
- Unannounced changes that break compatibility may occur if there is no way around it. Note this fact.
- Before use, it is probably a good idea to perform some acceptance tests to determine performance criteria including whether the package works as expected, is sufficiently fast, and has memory consumption within the acceptable limits. Creating acceptance tests up front also helps to avoid trouble when upgrading
nlib
. It is a good idea to use the testing
library to create these types of acceptance tests.
- The bug may be quickly fixed if you describe the code reproducing the bug as a test code in the
nn::nlib::testing
format and include it in your bug report.
nlib Package Description
- Note that the included libraries and supported platforms vary depending on the package.
Directory Structure
- The package is structured as follows.
├── bin
│ └── cmake # Command-line tools
├── cmake # Common Cmake scripts
├── documents-en
│ └── API # Reference Manual
│ ├── html # HTML version of the Reference Manual
│ └── searchdata.xml # Search index
├── include # Include files (in `system/include` for Cafe)
│ ├── nn
│ │ └── nlib
│ │ ├── exi
│ │ ├── heap
│ │ ├── msgpack
│ │ │ └── jsonrpc
│ │ ├── oss
│ │ ├── simd
│ │ ├── succinct
│ │ ├── testing
│ │ ├── threading
│ │ └── unicode
│ ├── lz4.h
│ ├── lz4hc.h
│ ├── lz4frame.h
│ ├── sqlite3.h
│ └── sqlite3ext.h
├── lib # Platform-specific library files (`system/lib` for Cafe, `libraries` for CTR)
├── samples # Source code for samples
│ ├── cmake
│ └── sources
│ ├── exi
│ ├── heap
│ ├── misc
│ ├── msgpack
│ ├── oss
│ ├── succinct
│ └── testing
└── setup
├── setup-bash-ubuntu-windows-tools.sh
├── setup-cygwin-tools.sh
└── setup-msvc-libraries.sh
├── setup-msvc-libraries.ps1
└── setup-vagrant-plugins.sh
Platform-Specific Descriptions
-
Descriptions of Each Library
- libn Library (including libn.a, libn.so, n-vc120amd64.lib, and n-vc120dllamd64.dll, etc.)
- This library includes functions that have C linkages for suppressing differences among various platforms.
- Specifically, the following functions are provided:
-
Functions, including
nlib_epochtime()
, that obtain the current time.
-
The
nlib_gen_random()
function that obtains random values.
-
Allocation and deallocation of (virtual) memory from the operating system
-
Functions and macros supporting pthread
-
Message queues supporting lock-free behavior (
nlib_mq
)
-
Functions, including
nlib_atomic_compare_exchange32()
, that atomically rewrite memory.
-
Functions, including
nlib_debug_break()
and nlib_debug_backtrace()
, that access the debugger.
-
The
nlib_getenv()
function that obtains environment variable values.
-
File handles and file I/O functionality, including
nlib_fd_open()
.
-
Functions relating to string searches, accelerated with SIMD.
-
String encoding conversion between UTF-8 and UTF-16 and between UTF-8 and UTF-32, in both directions
-
Functions, including
nlib_strto_int32()
, that wrap standard C functions, including strtol()
and strtoll()
, in a way they are not wrongly used.
-
Functions, including
nlib_isspace()
, that implement standard C functions defined by ctype.h
in a way they are not affected by the locale.
-
Functions, including
nlib_snprintf()
, that are adjusted to suppress differences among the environment-specific behaviors of snprintf()
, snwprintf()
, and other similar functions to prevent them from being wrongly used.
-
Fast conversion of endianness (including
nlib_swapendian_32()
)
-
Fast CRC32 and CRC32C calculations (
nlib_crc32()
and nlib_crc32c()
)
-
Fast pop count calculations (including
nlib_popcnt64()
)
-
Macros that absorb differences among the various attributes for each compiler
- misc Library (libnx_misc.a, libnx_misc.so, nx_misc-vc120amd64.lib, nx_misc-vc120dllamd64.dll, etc.)
- This library is a class library that includes relatively small classes. It implements many features that are believed to be commonly used in programs.
- Specifically, it includes the following.
-
Base stream classes that serve as a foundation for
InputStream
and OutputStream
-
Classes that read and write UTF-8 strings from streams while validating (
TextReader
and TextWriter
)
-
Classes that read and write binary data from streams (
BinaryReader
and BinaryWriter
)
-
Classes for handling Unicode
-
Classes and functions for handling threads
-
Classes and functions for handling date and time information
-
Classes for adding and subtracting dates and times, in addition to parsing and creating RFC 2822 or W3C DTF date strings (
DateTime
and TimeSpan
)
-
Classes and functions for handling URIs
-
URI parsers (
Uri
)
-
Classes for processing RFC 6570 URI templates (
UriTemplate
)
-
A class for standardizing file paths, which are often represented by different strings depending on the environment, into URI strings (
NativePathMapper
).
-
A class for handling asynchronous file access (
threading::AsyncFileIo
)
-
Basic lock-free data structure and the algorithm
-
The different types of gadget classes
-
A substitute class for
std::unique_ptr
(UniquePtr
). In C++11, this class is defined as std::unique_ptr
using a typedef
.
-
Singleton classes (
Singleton
and SimpleSingleton
)
-
An
std::vector
-like class that does not require freeing memory and copying elements upon expansion (Nlist
)
-
A class for parsing command-line strings (
Nflags
)
-
A class for supporting handle creation (
HandleMaker
)
The namespaces include nn::nlib
, nn::nlib::unicode
, and nn::nlib::threading
.
- exi Library (libnx_exi.a, libnx_exi.so, nx_exi-vc120amd64.lib, nx_exi-vc120dllamd64.dll, etc.)
- A library used to read and write binary XML. It can also read and write text-based XML. It supports XML namespaces and includes a simple, easily programmable pull parsing API.
The namespace is nn::nlib::exi
.
- heap Library (libnx_heap.a, libnx_heap.so, nx_heap.lib, nx_heap-vc120dllamd64.dll, etc.)
- This library is a heap library for which memory allocation and deallocation is optimized with multithreaded operations. The
nmalloc
and nfree
functions (and the heap::CachedHeap
and heap::CentralHeap
classes used to implement them) keep performance degradation in check by controlling fragmentation through the use of multi-level free lists. By maintaining per-thread caches for small blocks of memory, memory can be allocated and freed without locking, which helps to limit the degradation of multithreaded performance. Pointers within the free lists are checked for both mangling and their allocation status. This implementation makes it more difficult to use them in an invalid way by overwriting memory externally.
The namespaces are nn::nlib::heap
and the global namespace (C linkage functions).
- succinct Library (libnx_succinct.a, libnx_succinct.so, nx_succinct-vc120amd64.lib, nx_succinct-vc120dllamd64.dll, etc.)
- A library used to handle succinct data structures. You can think of a succinct data structure as one whose content can be searched while it is still compressed.
The succinct
library includes the following.
-
Implementation of succinct vector classes
-
A class that implements balanced parentheses
-
A class that implements LOUDS (level-order unary degree sequences)
-
Trie implementation
-
A class for creating high-compression indices and searching strings using the Aho–Corasick algorithm
The namespace is nn::nlib::succinct
.
- msgpack Library (libnx_msgpack.a, libnx_msgpack.so, nx_msgpack-vc120amd64.lib, nx_msgpack-vc120dllamd64.dll, etc.)
- A library for reading and writing MessagePack and JSON. It also includes a class for parsing CSV.
The msgpack
library includes the following.
-
Classes for reading and writing JSON from streams
-
Classes for reading and writing MessagePack from streams
-
A class for high-speed in-memory MessagePack parsing
-
A class for reading CSV
-
Classes for storing JSON, MessagePack, and CSV data after it has been read
-
Implementation of JSON-RPC
The namespace is nn::nlib::msgpack
.
- testing Library (libnx_testing.a, libnx_testing.so, nx_testing-vc120amd64.lib, nx_testing-vc120dllamd64.dll, etc.)
- A library for writing highly portable test code. It is a testing framework that is somewhat source-code compatible with tests written for Google Test.
The namespace is nn::nlib::testing
. (It is aliased to the testing
namespace for the sake of Google Test compatibility.) It depends on the misc
library.
- oss Library (libnx_oss_bsdiff.a, libnx_oss_sqlite3.a, libnx_oss_lz4.a, etc)
- Contains code for open-source software (OSS), code derived from OSS, and code for accessing OSS. To use it, an OSS license must be processed.
The following libraries are currently included. They are built as separate library files.
-
A library conversion of bsdiff that has been modified to use
zlib
instead of bzlib2
, and the corresponding command-line programs
-
A version of SQLite3 that has been modified to run using the interfaces provided by
nlib
, and a corresponding command-line shell
-
LZ4 is approximately 20 times faster at data compression than
zlib
, and approximately five times faster at decompression.
- The namespaces are
nn::nlib::oss
and the global namespace (C linkage functions).
- simd Library (Mostly Inline Functions Included in the misc Library)
- SIMD instructions can significantly increase the speed of programs. The
simd
library makes it possible to write code that supports both SSE4.2 and NEON.
It supports integer and single-precision floating-point number calculations that enable your code to compute at high speeds.
Single-precision floating-point number calculation can be used even in environments that do not support SSE or NEON.
On Cafe, this is optimized to use paired singles.
- In other words, the same code can support the following environments.
-
On x86 processors, the code is compiled with speed boosts through the use of SSE 4.2.
-
On processors with NEON support, the code is compiled with speed boosts through the use of NEON.
-
On Cafe, the code is compiled with speed boosts through the use of paired singles. (Floating-point operations alone are quite a bit slower than the Cafe-SDK for operations like matrix calculations, because the interface returns values for data like four-dimensional vectors.)
-
On CTR, generic code is compiled (floating-point operations only).
The namespace is nn::nlib::simd
.
Dependency between nlib libraries.
The dependency between nlib
libraries is as shown in the following figure.
C++11 Features Used by nlib
nlib
is compiled using C++11 features for some combinations of compilers and versions. The table below summarizes the C++11 features, compatibility macros that can be used with nlib
, and the compilers that compile using C++11. Note that for clang
, the determination is made based on __has_feature
and __has_include
macros, and not the version.
- C++11 features are not used in any other environment.
C++11 Feature | Identification Macro | Macros and Typedefs for C++03 Compatibility | Visual Studio | gcc |
Rvalue references | NLIB_CXX11_RVALUE_REFERENCES | None | 2010 | 4.3 |
Variadic templates | NLIB_CXX11_VARIADIC_TEMPLATES | None | 2013 | 4.7 |
Static assertions | NLIB_CXX11_STATIC_ASSERTIONS | NLIB_STATIC_ASSERT() | 2010 | 4.3 |
Null pointer constant | NLIB_CXX11_NULL_POINTER_CONSTANT | nn::nlib::nullptr, nn::nlib::nullptr_t | 2010 | 4.6 |
Deleted functions | NLIB_CXX11_DEFAULTED_AND_DELETED_FUNCTIONS | NLIB_DISALLOW_COPY_AND_ASSIGN() | 2015 | 4.4 |
Explicit conversion operators | NLIB_CXX11_EXPLICIT_CONVERSION_OPERATORS | NLIB_SAFE_BOOL() | 2013 | 4.6 |
Alias templates | NLIB_CXX11_TEMPLATE_ALIAS | None | 2013 | 4.7 |
constexpr | NLIB_CXX11_CONSTEXPR | NLIB_CEXPR | 2015 | 4.7 |
char16_t, char32_t | NLIB_CXX11_NEW_CHARACTER_TYPES | nlib_utf16_t, nlib_utf32_t | 2015 | 4.4 |
alignas | NLIB_CXX11_ALIGNMENT_SUPPORT | NLIB_ALIGNAS() | 2015 | 4.8 |
Explicit virtual overrides | NLIB_CXX11_EXPLICIT_VIRTUAL_OVERRIDES | NLIB_OVERRIDE | 2012 | 4.7 |
final | NLIB_CXX11_EXPLICIT_VIRTUAL_OVERRIDES | NLIB_FINAL | 2012 | 4.7 |
noexcept | NLIB_CXX11_NOEXCEPT | NLIB_NOEXCEPT | 2015 | 4.6 |
Range-based for | NLIB_CXX11_RANGE_BASED_FOR | None | 2012 | 4.6 |
lambda | NLIB_CXX11_LAMBDAS | None | 2012 | 4.7 |
Delegating constructors | NLIB_CXX11_DELEGATING_CONSTRUCTORS | None | 2013 | 4.7 |
std::unique_ptr | NLIB_CXX11_UNIQUEPTR | nn::nlib::UniquePtr | 2013 | 4.7 |
#include <random> | NLIB_CXX11_STDLIB_RANDOM | None | 2013 | 4.8 |
#include <chrono> | NLIB_CXX11_STDLIB_CHRONO | Conversion to and from nn::nlib::DateTime | 2012 | 4.7 |
#include <atomic> | NLIB_CXX11_STDLIB_ATOMIC | NLIB_MEMORY_ORDER_SEQ_CST and others | 2012 | 4.8 |
#include <array> | NLIB_CXX11_STDLIB_ARRAY | None | 2012 | 4.5 |
#include <unordered_map>
#include <unordered_set> | NLIB_CXX11_STDLIB_UNORDERED | None | 2012 | 4.5 |
#include <tuple> | NLIB_CXX11_STDLIB_TUPLE | None | 2012 | 4.5 |
nlib Coding Rules
nlib Basic Coding Rules
- The
nlib
coding rules are a slight modification and relaxation of the latest version of the Google C++ Style Guide (and a slightly outdated Japanese translation).
- The reasons for using the Google C++ Style Guide are listed below.
-
It is easy to access documentation about the coding rules. Translations are available (albeit unofficial ones).
-
Similar to the general coding rules for the gaming industry, C++ exceptions and RTTI use are prohibited.
-
It has been updated continuously in the past.
-
Changes made to the coding rules with each update are reasonable, and there haven't been any sudden, drastic changes.
-
We can expect it to be maintained even five years from now.
- The changes and relaxations adopted for
nlib
are shown below.
Coding rule | Google C++ Style Guide | nlib |
Line length | 80 | 100 |
Indentation | 2 | 4 |
Parameter order | Inputs followed by outputs | Outputs followed by inputs |
Default argument | Prohibited in principle | Allowed for use except for the cases described below:
-
In the case of a constructor. Issues may arise if the default value allows an argument not to be explicitly specified until only one argument remains unhandled.
-
In the case of a virtual function. Issues may arise if the overridden function has a default argument different from that of the original function.
-
In the case where the argument can be used as a function pointer. Confusion may be induced in the coding.
|
File name | file_name.cc | FileName.cpp(file_name.cpp in the sample) |
#include guard | | Also append #pragma once (for accelerating compilation in Visual Studio) |
Use of boost | | Not to be used by libraries (may be used for other purposes such as testing) |
Accessor function name | get_xyz() / set_xyz() | GetXyz() / SetXyz() |
Argument passed by reference | Non-const references are prohibited | In nlib , non-cost references are allowed as exceptions in the following cases.
-
When specializing
std::swap
-
Code that imitates move semantics in C++03
-
Class with simple features. For example, a class having a feature that only maintains a lock, such as the
ScopedLock class.
-
When passing
nn::nlib::UniquePtr as an argument (Returning a value would be inefficient, and it would be strange to pass a pointer to UniquePtr .)
|
Use of C++11 | | Ensure that headers and source code can be properly compiled also by using C++03 compilers.
-
This doesn't mean you can't use C++11. Features such as move constructors, range-based for statements, explicit virtual overrides, and explicit conversion operators should all be supported proactively.
-
If possible, the same source code is compiled using macros and other preprocessor directives. If not possible, implement similar features that can be used from C++03.
|
nullptr | Use if C++11 | In principle, don't use nullptr_t or nullptr as long as C++03 is used.
-
The
nullptr idiom for implementing nullptr in C++03 results in errors if it is defined in multiple libraries, so it cannot be defined in the global namespace.
-
With
nlib , it is defined as nn::nlib::nullptr_t and nn::nlib::nullptr for C++03.
-
Avoiding the use of misleading overloads is a workaround for most cases that would require
nullptr .
|
constexpr | Use if C++11 | For the time being, use of constexpr should be avoided except in obvious cases.
-
Unfortunately, code that uses
constexpr proactively has a drawback in terms of portability, because differences among compiler behavior can result in many compilation errors.
-
"Obvious cases" refers to null constant objects or cases that one would expect to compile without issues on any compatible compiler that supports
constexpr (for example, simple constructors that merely set constants).
|
Comment | |
-
Considering translation costs, do not use non-ASCII (non-English) characters in comments for public headers (This rule does not apply if the content should be never translated).
-
If you want to write detailed comments, include them in either source code files or the Reference Manual.
-
Doxygen comments must not be written in public headers. This causes recompilation due to changes in the documentation, and increases the translation burden. Files such as
manual_ja.h and manual_en.h should be prepared for each library and comments should be written into those files.
-
In the reference manuals, never place newline characters in the middle of a sentence. This rule is meant to support the nature of the translation-aid software used for localization. It takes precedence over the previously mentioned limit of 100 characters per line.
|
Checking and Correcting Coding Style
- The coding style can be checked easily by using cpplint.
- The warning criteria have been slightly relaxed, as follows.
-
Specify
--linelength=100
to allow up to 100 characters per line.
-
Warnings due to
cpplint
's legal/copyright
, build/include_alpha
, and whitespace/newline
categories have been suppressed.
-
Includes of the
algorithm
header are not required when using std::swap
.
- You can run checks easily by writing rules like the following in your makefile.
cpplint:
-./cpplint.py --linelength=100 \
$(shell find ./include/ -path "*.h") \
$(shell find ./sources/ -path "*.cpp" -or -path "*.h") 2>&1 | tee cpplint.log
- In addition,
nlib
supports checking source code using clang-check
. Generating a makefile with CMake creates a file called compile_commands.json
, which clang-check
uses to run.
- For more information, see JSON Compilation Database Format Specification in the
clang
manual.
Automatic Formatting of Coding Style
- You can also use
clang-format
to standardize the indentation and other stylistic aspects to suit your preferences. You can use the configuration used by nlib
, which is stored in a .clang-format
file (for clang
3.5) that is included in the nlib
package. It is recommended that you first convert your source code en masse using clang-format
, and then use cpplint.py
to do common detailed checks.
Setting Warnings During Compilation
- Code quality can be improved by testing with multiple compilers and runtime environments. Configure all compilers to relatively strict warning levels.
-
Ensure successful compilation with Visual Studio 2013 and all newer versions. In general, resolve all warnings shown when using the /W4 option.
-
Suppression of
pragma
warnings is permitted.
-
Make sure that the latest updates are always applied, and check
_MSC_FULL_VER
in your code.
-
Periodically reduce warnings using code analysis in Visual Studio.
-
Make sure that compilation succeeds using the latest versions of
gcc
and clang
binary packages available for Cygwin (32-bit) and Ubuntu. In general, resolve all warnings shown when using -Wall -Wextra.
-
Always use the newest version of Cygwin. If a Cygwin update causes a build not to work as expected, revise the build scripts.
-
Suppression of
pragma
warnings is permitted.
-
Use
-Wstack-usage=65536
to ensure that the stack is not overused.
-
Use
-fstack-protector
.
-
Make sure that compilation succeeds without warnings in the CAFE development environment.
-
Make sure that compilation succeeds in the CTR development environment.
-
It is preferable to have compilation succeed in all 64-bit environments.
-
Also resolve all Doxygen warnings.
- The basic release conditions for
nlib
are ensuring that there are zero cpplint.py
warnings and that compilation and tests succeed in all environments.
Additional Rules and Conventions
-
In general, code for functions that are not declared or implemented in functions defined in
Platform.h
must appear within code that can be compiled in all environments it could conceivably be used in. That said, inclusion of code that is optimized for specific hardware using SIMD instructions or other such techniques is permitted.
-
Use macros like
NLIB_FINAL
or NLIB_OVERRIDE
and typedef
s to safely incorporate C++11 features and continuously make it possible to port to C++11 or C++14.
-
Use POSIX.1 2008 error codes for error values whenever possible. Use the
errno_t
type, as recommended by https://www.securecoding.cert.org/confluence/x/QgAy.
For example: Using code like this makes it easier to understand that these are error values.
-
Values of POSIX.1 2008 error codes are sufficiently small integer values, so immediate values can be loaded efficiently on processors like ARM.
-
They must be implemented without using exceptions or RTTI. Make sure that your application code (that calls
nlib
) is written in a way that anticipates the use of these features, such as exception safety.
-
Instead of functions like
malloc
and free
, use functions like nlib_malloc
and nlib_free
.
-
Use
new (std::nothrow)
for new
.
-
The global
new
operator must not be overloaded within the library.
-
This overloading is not allowed because it affects the behavior of other libraries that you wouldn't think of as having dependencies.
-
Encode multi-byte strings in either ASCII or UTF-8.
-
The implementation must allow UTF-8, including in Windows. Path strings and strings passed to
nlib_printf
are passed to Win32 API after conversion to UTF-16 in nlib
.
-
Command-line argument processing (
Nflags
) is an exception.
-
The
char
type must work correctly, whether signed or unsigned.
-
These can be handled explicitly as signed values by using
signed char
.
-
UTF-16 is supported if
wchar_t
is 16-bit; UTF-32 is supported if wchar_t
is 32-bit.
-
NLIB_WCHAR_SIZE
is defined as either 2 or 4, so you can use this to make the determination.
-
Use
nlib_utf16_t
as the type for UTF-16 characters, and use nlib_utf32_t
as the type for UTF-32 characters.
-
In C++11, these types are defined with
typedef
as char16_t
and char32_t
, respectively.
-
In general, prioritize UTF-8 over UTF-16 and UTF-32.
-
UTF-8 consumes less memory, and is easy to support on multiple platforms.
-
The existence of surrogate pairs and combining characters make it impossible to always represent a single character with a fixed length, even if using UTF-16 or UTF-32.
-
Using
nlib_strcplen_ex
allows you to request the number of code points and byte length simultaneously while performing validation. If SIMD is supported, this operation can be sped up even more.
-
Move constructors and move assignment operators must be defined in code separated by
NLIB_CXX11_RVALUE_REFERENCES
, and you must also define member functions like the following so that the same functionality can be implemented in C++03. Excellent alternative approaches like boost::move
exist, but they cannot achieve full compatibility, so they are not used.
-
type_traits
are used with care. This feature of C++11 is sometimes buggy or is not a complete implementation for some compilers or compiler versions. A tentative wrapper has been provided in nn/nlib/TypeTraits.h
, but only use it if there is no other alternative.
-
Do not use non-copyable mixins.
-
Use
NLIB_DISALLOW_COPY_AND_ASSIGN
instead.
-
The increase in levels of inheritance can sometimes have a negative effect on source code analysis programs and documentation systems.
-
For the C standard library headers, include files in
.h
format. There are two reasons for this.
-
It handles situations when the C++ standard library headers are corrupted.
-
It minimizes code confusion caused by the inclusion of both
%<xxx.h%>
and %<cxxx%>
formats within nlib. (The %<xxx.h%>
format must be included because there is a requirement that nn/nlib/Platform.h
can be compiled with a C compiler.)
-
Write tests in Google Test format, and make sure unit tests have been run on all platforms.
-
This makes it easier to find bugs that only occur in certain environments, in addition to platform-specific implementation bugs.
-
It is best if you separate the common tests to be run on any platform from the PC-specific tests.
-
This makes it easier to write stress tests and brute-force tests in environments with as much power as a PC.
-
On a PC environment, it is sometimes easy and convenient to use open-source software (OSS) in test programs.
-
In a Windows environment, DirectX and other such APIs can be used in test programs, which is sometimes convenient.
-
In environments where Google Test does not run, the
testing
library can be used.
-
When compiling using
gcc
, you can calculate test coverage automatically by using gcovr
.
-
Write the Reference Manual in Doxygen format.
-
Doxygen is continuously being improved, and it is a documentation system that we can expect to exist and be improved even five years from now.
-
Doxygen comments are written within
manual.h
files located within each include directory. English versions of the manual are stored as manual_en.h
. This approach allows multiple languages to be managed within the same repository.
-
If possible, enable building using CMake.
-
CMake is a representative selection for cases when you are coding in C/C++ that must run on both Windows and Linux.
Introduction to CMake
- CMake can be used for building samples. CMake has been well-known for being adopted to build MySQL (MariaDB). Currently, it has established the position as the standard build system for various software supporting both Windows and Linux because it is written in C/C++. You may have noticed that many OSSs contain the file
CMakeLists.txt
. Using CMake allows you to generate makefiles for GNU make, Visual Studio and Xcode projects by writing this CMakefiles.txt
script file.
- The main advantages of using CMake are as follows.
-
It supports many systems and compilers.
-
It supports Windows, Linux, Mac OS X, FreeBSD, IRIX, AIX, Sun OS, HP-UX, and more.
-
It supports multiple IDEs, including all the Visual Studio versions, Eclipse CDT and Xcode.
-
You can also use Express Edition of Visual Studio and each of the latest versions of Visual Studio will be ready to be supported by the time when its RC version is released.
-
It can support cross-compilation with relatively simple settings.
-
Temporary files and other files created during builds don't clutter up the directories where the source files are located. This means Out-Of-Tree builds are possible. This function is always required when binary for multiple environments needs to be generated from one source tree.
-
Multiple independent projects can be generated from CMake and then used, which makes parallel builds easy.
-
It searches for the locations of required libraries and tools automatically.
-
The include path settings for libraries and links to libraries have been made cross-platform, so it is possible to write unified code even if the paths vary for each environment.
-
Installers can be created (
CPack
).
-
NSIS, deb, rpm, tgz, and so on.
-
Tests can be run and managed (
CTest
).
- With
nlib
, using CMake allows you to generate solution files that work with all Visual Studio versions, makefiles for UNIX and Cygwin, and so on, and then build the program.
- The
./cmake/toolchain-cafe.cmake
and ./cmake/toolchain-ctr.cmake
files within the package are configured for cross-compilation. This makes it possible to use CMake to generate makefiles even for CAFE/CTR sample programs, and then build the programs. The CAFE and CTR versions of nlib
also have scripts for the standard make
and omake
build systems.
Cautions When Using CMake
-
A Windows version of CMake must be included to generate Visual Studio projects.
-
A Cygwin version of CMake must be installed to generate makefiles for Cygwin.
-
With CMake, Visual Studio and Xcode projects and makefiles must be output to separate directories for each Debug build and Release build.
FAQ
- This section explains commonly encountered problems while using
nlib
, and their workarounds.
- Overloading new/delete
nlib
uses new
(std::nothrow
) for memory allocation. When applications and other libraries overload new
or delete
, operator new(size_t, const std::nothrow_t&)
must also be overloaded.
- For more information, see the
heap_replace_malloc
sample.
- Overriding Functions Like malloc
nlib
does not use malloc
or free
directly. Instead, it uses functions like nlib_malloc
and nlib_free
. These functions are weakly defined, so they can be overridden by user-defined functions.
- You must override these functions if you want to use a user-defined heap. The following functions are required.
- For more information, see the
heap_replace_malloc
sample.
- Compile Options
- Platform-specific switches and other such definitions are defined in files such as
cmake/common.cmake.win32
within the package. Please reference them.
- Compilation results in "MACRO_XXXX not defined, compile may fail" being displayed.
- You need to define
MACRO_XXXX
using the command-line arguments to the compiler.
- Should we use the latest version for our project?
- That's best left up to the judgment of the project using
nlib
. Writing acceptance tests for future use makes it easier to make this type of decision. However, only the latest version receives support. Note this fact.
- Please release a special version of the package for our project.
- There are as many project requirements as there are projects. For example, in the past, inquiries like the following have been received.
-
We don't need library X, so we'd like to have a special release with that removed.
-
We'd like a release without any samples.
-
We are using the 64-bit version of Visual Studio 2015, so we'd like a release that comes with a pre-built Visual Studio 2015 solution.
To handle such requirements in a flexible manner, the projects using nlib
are permitted to restructure or repackage nlib
and to redistribute it, but only within the project.
- Why can't text encoded using Shift-JIS be displayed with nlib_printf?
- The input string for
nlib_printf
must be encoded using UTF-8.
- There is a bug in a version released half a year ago. Will you please release a patch that fixes it?
- Bug fixes are applied to the latest version. You need to use a newer release that incorporates the fix.
- Will you please make it follow our library or framework's approach?
- If your approach is publicly available, and is supported by all platforms (Windows, Linux, FreeBSD, OS X, Cafe, and CTR), this might be possible. Please get in touch.
Revision History
2016-12-22
-
Added libraries, compiled for Visual Studio 2017 RC, to the package.
-
For building samples for Visual Studio, added a script,
gen-sln.ps1
, for creating a Visual Studio solution file from PowerShell using CMake.
-
For building samples for CAFE and CTR, added scripts,
cmake_cafe_ninja.ps1
and cmake_ctr_ninja.ps1
, for creating a build script for ninja from PowerShell using CMake.
-
On and after this release, libraries compiled for Visual Studio 2012 will be no longer included in the package.
-
Added the new function
nlib_pause()
.
-
Made other minor changes and fixes.
2016-11-29
-
On and after the next release, libraries compiled for Visual Studio 2012 will no longer be included in the package.
-
Started operation check in Fedora 25. Operation check in Fedora 24 is no longer supported.
-
Defined the new macro
NLIB_TLS_INVALID
.
-
Added the new function
nlib_atomic_exchangeptr()
.
-
SIGLO-41003: Fixed an issue where dumping the heap status after allocating a large size of memory caused a crash.
-
NXBTS-8006: Fixed an issue where signed 64-bit integers of msgpack may be incorrectly parsed in the msg library.
-
Fixed an issue where an crash may occasionally occur due to issues related to implementation of DllMain for Windows libraries.
-
nlib_tryonce()
is now deprecated. This function will be deleted in a future release.
-
The /sdl option is now set during compilation with Visual Studio.
-
Made other minor changes and fixes.
2016-10-25
2016-09-20
-
Started operation check in FreeBSD 11.0. FreeBSD 10.3 is no longer officially supported.
-
Updated the version of SQLite bundled with the
OSS
libraries to 3.14.1.
-
Implemented
nlib_bitreverse32()
and nlib_bitreverse64()
.
-
It is now possible to specify
O_EXCL(NLIB_FD_O_EXCL)
in nlib_fd_open()/tt>.
-
Modified nlib_fd_create()
so that it fails if the file already exists.
-
nlib_strcplen2()
was renamed nlib_strcplen()
. nlib_strcplen2()
will be provided as a macro for a while and then be deleted.
-
nlib_popcnt()
was renamed nlib_popcnt32()
. nlib_popcnt()
will be provided as a macro for a while and then be deleted.
-
nlib_clz()
was renamed nlib_clz32()
. nlib_clz()
will be provided as a macro for a while and then be deleted.
-
nlib_ctz()
was renamed nlib_ctz32()
. nlib_ctz()
will be provided as a macro for a while and then be deleted.
-
Deleted deprecated files and functions.
-
Made other minor changes and fixes.
2016-08-23
-
Started operation check on Bash on Ubuntu on Windows. The compiled libraries are shipped with your PC version package.
-
Added
heap::CentralHeap::FreeWithSize()
, heap::CentralHeap::GetAllocSize()
, and heap::CentralHeap::Realloc()
.
-
Added
NMALLOC_QUERY_SET_COLOR
, NMALLOC_QUERY_GET_COLOR
, NMALLOC_QUERY_SET_NAME
and NMALLOC_QUERY_GET_NAME
to queries for nmalloc_query()
. This function associates a 24-bit integer value or a string with any allocated region of 4096 bytes or more.
-
For how to use this function, see the
object_tracking
sample in the heap
library.
-
Added
NMALLOC_DUMP_PAGESUMMARY
to NMallocDumpMode
.
-
Improved improper behaviors of commands, including
nmalloc
, when failing to allocate a TLS index. nmalloc_setmark1()
, nmalloc_setmark2()
, nmalloc_getmark()
and nmalloc_getobjptr()
are now deprecated. They will be deleted in a future release.
-
NMALLOC_HEAPOPTION_CHECK_0
and NMALLOC_HEAPOPTION_CHECK_1
are now deprecated. They will be deleted in a future release.
-
Made other minor changes and fixes.
2016-07-26
2016-06-29
-
The compatibility of the source code is partially lacked in this release.
-
Started operation check in Fedora 24. Operation check in Fedora 23 is no longer supported.
-
Started operation check in Alpine Linux 3.4. Operation check in Alpine Linux 3.3 is no longer supported.
-
The C linkage functions, including
nlib_sleep()
, that have been declared in the misc
library in the global namespace are now moved from that library to the new libn
library.
-
A 64-bit cygwin build is provided as its PC version, and the library directory is changed from
lib/cmake/gcc
to lib/cmake/cygwin
.
-
Added
nlib_isalnum()
, nlib_isalpha()
, nlib_iscntrl()
, nlib_isdigit()
, nlib_isgraph()
, nlib_islower()
, nlib_isprint()
, nlib_ispunct()
, nlib_isspace()
, nlib_isupper()
, nlib_isxdigit()
, nlib_tolower()
, and nlib_toupper()
.
-
Deprecated the functions, including
IsAlnum()
, implemented as C++ functions.
-
Added
nlib_strto_int32()
, nlib_strto_int64()
, nlib_strto_uint32()
, nlib_strto_int64()
, nlib_strto_double()
, nlib_strto_float()
, nlib_strto_int32_fallback()
, nlib_strto_int64_fallback()
, nlib_strto_uint32_fallback()
, nlib_strto_int64_fallback()
, nlib_strto_double_fallback()
, nlib_strto_float_fallback()
, nlib_strto_int8()
, nlib_strto_int16()
, nlib_strto_uint8()
, and nlib_strto_uint16()
.
-
Deprecated
StrTo()
and StrToFallback()
implemented as C++ functions.
-
Added
nlib_epochtime_timespec()
, nlib_ticktime_timespec()
, nlib_sleep_timespec()
, nlib_mutex_trylock_for_timespec()
, nlib_semaphore_trywait_for_timespec()
, nlib_cond_wait_for_timespec()
, nlib_cond_wait_until_timespec()
, nlib_rwlock_tryrdlock_for_timespec()
, nlib_rwlock_tryrdlock_until_timespec()
, nlib_rwlock_trywrlock_for_timespec()
, nlib_rwlock_trywrlock_until_timespec()
, nlib_condrwlock_wait_for_timespec()
, and nlib_condrwlock_wait_until_timespec()
.
-
The specification for the argument
nlib_fd_open()
is changed to change the last argument type from void*
to int
and specify access permission.
-
The specification for the argument
nlib_thread_setname()
is changed to allow you to specify a thread you want to change its name.
-
The behavior of
heap::CentralHeap(nmalloc)
of the heap
library has been further improved.
-
Deleted deprecated files and functions.
-
Made other minor changes and fixes.
2016-05-31
-
Starting with the next release, a 64-bit cygwin build is provided as its PC version, and the library directory will be changed from
lib/cmake/gcc
to lib/cmake/cygwin
.
-
Added
nlib_remove()
and nlib_thread_getname()
.
-
Improved the behavior of
heap::CentralHeap(nmalloc)
of the heap
library.
-
Mitigated the issue where allocating a large size of memory (1 MB or more) caused fragmentation.
-
Mitigated the issue where an assigned memory address range was larger than necessary.
-
Other minor changes and fixes
2016-04-26
-
Started operation check in Alpine Linux 3.3.
-
Started operation check in Ubuntu 16.04LTS. Ubuntu 15.10 is no longer officially supported.
-
Started operation check in FreeBSD 10.3. FreeBSD 10.2 is no longer officially supported.
-
Mitigated an issue where allocating relatively large size memory (4 KB or more) using
heap::CentralHeap
(nmalloc)
from the heap
library caused fragmentation.
-
Allocation of a memory size up to 1 MB can now best fit within a free list (previously a size up to 512 KB can do so).
-
Fixed an issue where memory is prone to have fragmentations if the memory is allocated with an alignment.
-
Updated the version of SQLite to 3.12.2, which is included in the
oss
libraries. In addition, temporary files can be now automatically deleted.
-
In
nlib_fd_pwrite()
and nlib_fd_pread()
for the CAFE and CTR versions, corrected the behavior found when an offset larger than the file size is specified.
-
simd::I64
is now deprecated.
-
The
Crc32
class is now deprecated. Use nlib_crc32()
or nlib_crc32c()
instead.
-
Other minor changes and fixes
2016-03-29
-
Ubuntu 12.04LTS has been verified for operation..
-
SSE4.2 is now required for execution of the PC version.
-
The
nlib_compiler_version()
function was added to the misc
library. It returns the compiler version used to compile nlib.
-
The
nlib_crc32()
and nlib_crc32c()
functions were added to the misc
library.
-
The following functions were added to the
simd
library.
-
simd::I128::LoadLoA8(), simd::I128::LoadLoA4(), simd::I128::LoadLoA2(), simd::I128::LoadLoA1()
-
simd::I128::LoadHiA8(), simd::I128::LoadHiA4(), simd::I128::LoadHiA2(), simd::I128::LoadHiA1()
-
simd::I128::StoreLoA8(), simd::I128::StoreLoA4(), simd::I128::StoreLoA2(), simd::I128::StoreLoA1()
-
simd::I128::StoreHiA8(), simd::I128::StoreHiA4(), simd::I128::StoreHiA2(), simd::I128::StoreHiA1()
-
simd::I128::Permute8(), simd::I128::Permute16(), simd::I128::Permute32()
-
simd::I128::SetMask8(), simd::I128::SetMask16(), simd::I128::SetMask32()
-
simd::I128::Test8(), simd::I128::Test16(), simd::I128::Test32()
-
simd::I128::CmpEq64(), simd::I128::CmpLtInt64(), simd::I128::CmpGtInt64(), simd::I128::CmpLtUint64(), simd::I128::CmpGtUint64()
-
simd::I128::CmpLeInt64(), simd::I128::CmpGeInt64(), simd::I128::CmpLeUint64(), simd::I128::CmpGeUint64()
-
simd::I128::CmpEqZero8(), simd::I128::CmpEqZero16(), simd::I128::CmpEqZero32(), simd::I128::CmpEqZero64()
-
simd::I128::ConvertFromInt8ToInt16Lo(), simd::I128::ConvertFromInt8ToInt16Hi()
-
simd::I128::ConvertFromInt16ToInt32Lo(), simd::I128::ConvertFromInt16ToInt32Hi()
-
simd::I128::ConvertFromInt32ToInt64Lo(), simd::I128::ConvertFromInt32ToInt64Hi()
-
simd::I128::ConvertFromUint8ToUint16Lo(), simd::I128::ConvertFromUint8ToUint16Hi()
-
simd::I128::ConvertFromUint16ToUint32Lo(), simd::I128::ConvertFromUint16ToUint32Hi()
-
simd::I128::ConvertFromUint32ToUint64Lo(), simd::I128::ConvertFromUint32ToUint64Hi()
-
simd::I128::Zip8Hi(), simd::I128::Zip8Lo(), simd::I128::Unzip8Hi(), simd::I128::Unzip8Lo()
-
simd::I128::Zip16Hi(), simd::I128::Zip16Lo(), simd::I128::Unzip16Hi(), simd::I128::Unzip16Lo()
-
simd::I128::Zip32Hi(), simd::I128::Zip32Lo(), simd::I128::Unzip32Hi(), simd::I128::Unzip32Lo()
-
simd::F128::CmpEqZero(), simd::F128::CmpNeZero(), simd::F128::CmpNearEqZero()
-
simd::F128::CmpLtZero(), simd::F128::CmpLeZero()
-
simd::F128::CmpGtZero(), simd::F128::CmpGeZero()
-
simd::F128::ConvertFromFixedPoint(), simd::F128::ConvertToFixedPoint()
-
Improved the execution speed of the NEON version code provided in the
simd
library.
-
Deleted deprecated files and functions.
-
Made other minor changes and fixes.
2016-02-23
-
The
oss
library now uses nlib_malloc
instead of nmalloc
in platforms other than CAFE and CTR.
-
NMALLOC_QUERY_UNIFY_FREELIST
can now be specified for the argument of nmalloc_query()
in the heap
library.
-
The fragmentation of metadata in the heap retained by the
heap
library's CentralHeap
is reduced.
-
Added
simd::I128::PopCntMask8()
, simd::I128::ClzMask8()
, and simd::I128::CtzMask8()
to the simd
library.
-
Corrected issues where some header files were missing.
-
Fixed an issue where
nlib_malloc_size()
could not be used in the Windows version of the DLL.
-
Made other minor changes and fixes.
2016-01-26
-
Started operation check in CentOS 7.2. Operation check in CentOS 7.1 is no longer supported.
-
Started operation check in Fedora 23. Operation check in Fedora 22 is no longer supported.
-
Added the
nlib_is_error()
function. It allows errors to be definitely evaluated and described for the return values of errno_t or bool.
-
The
nlib_getenv()
function now supports CAFE and CTR.
-
The
IsOk()
and IsError()
functions for each class have been depreciated. Use the nlib_is_error()
or operator bool()
function instead.
-
Added the
Init()
member function to Utf16InputStream
, Utf32InputStream
, WcharInputStream
, Base64InputStream
and Base64OutputStream
in the misc
library.
-
The initialization process for the
TextReader
, TextWriter
, BinaryReader
and BinaryWriter
classes for the misc
library is now split into two stages: one with Init()
and one with Open()
.
-
The
IsOk()
, IsError()
, Initialize()
and GetErrorValue()
member functions for each class in the exi
library have been depreciated.
-
The
Initialize()
member function for each class in the heap
library has been depreciated.
-
The initialization process for the
CsvReader
class for the msgpack
library is now split into two stages: one with Init()
and one with Open()
.
-
Fixed an issue where
nmalloc_aligned()
crashed if memory was assigned with an alignment larger than 4096 bytes.
-
Corrected the issue where
nmalloc_query(NMALLOC_QUERY_MAX_ALLOCATABLE_SIZE, ...)
could return a size larger than the one actually allocated.
-
Corrected the issue where
F128::StoreLoA4()
and F128::StoreHiA4()
of the simd
library may crash during operation on the ARM processor.
-
Made other minor changes and fixes.
- Contains Past Revision Histories.