Use the libraries and classes that support each input device to easily incorporate into your application the input from the various input devices on the system, such as the digital buttons, Circle Pad, touch panel, accelerometer, gyro sensor, microphone, and cameras.
6.1. Libraries That Use Devices
To access the input and other devices on the system from your application, you must use the libraries provided by CTR-SDK. The functions in many of the libraries for using devices return an instance of the nn::Result
class.
Call the nn::Result
class member function IsSuccess
or IsFailure
to find out whether the function completed processing successfully. If IsSuccess
returns true
, the function completed successfully; if it returns false
, the function failed. The IsFailure
return values have the opposite meaning; where true
means the function failed, and false
means the function completed successfully.
The nn::Result
class records error severity, a description, and the name of the module that raised the error. In addition to checking for function success, you may sometimes need to use this detailed information to find a way to work around or fix an error in your code.
6.2. HID Library
Using the HID library, you can handle input from the digital buttons (+Control Pad, A/B/X/Y/L/R Buttons, and START), Circle Pad, touch panel, accelerometer, gyro sensor, debug pad, and Circle Pad Pro.
Call the nn::hid:Initialize
function to initialize the HID library. Successive calls to the initialization function do nothing and return an error.
After initialization, input is sampled automatically, and you can obtain input from each device based on the corresponding class. The timing and cycle of the input sampling start differ depending on the type of device. To end sampling, the class created by the acceleration and gyro sensors is destroyed. Call the nn::hid::ExtraPad::StopSampling
function when done to finalize the Circle Pad Pro, and for all others, finalize the HID library.
The C Stick, ZL Button, and ZR Button on SNAKE hardware can be handled the same way as a CTR unit with the Circle Pad Pro always attached. In this case, the C Stick corresponds to the Right Circle Pad, and the ZL and ZR Buttons correspond to the same buttons on the Circle Pad Pro.
For information about the C Stick and differences with the Circle Pad Pro, see 6.2.7. C Stick.
Type |
Class Handling Input |
Sampling Start Trigger |
Cycle |
---|---|---|---|
Digital buttons and Circle Pad |
|
HID library initialization |
4 ms |
Touch panel |
|
HID library initialization |
4 ms |
Accelerometer |
|
Reader class creation |
Avg. 10 ms |
Gyro sensor |
|
Reader class creation |
Avg. 10 ms |
Debug pad |
|
HID library initialization (when connected) |
16 ms |
Circle Pad Pro |
|
|
8–32 ms |
C Stick, ZL / ZR Button | nn::hid::ExtraPadReader |
ExtraPad class StartSampling function |
8–32 ms |
The library lets you set a sampling frequency of 8-32 ms for the C Stick, ZL Button, and ZR Button on SNAKE hardware, but we recommend setting a sampling frequency of 10-21 ms to match the hardware capabilities.
Call the nn::hid::Finalize
function to finalize use of the library. When using the nn::hid::ExtraPadReader
class, you must first call the nn::hid::ExtraPad::Finalize
function.
6.2.1. Digital Buttons and Circle Pad
Call the nn::hid::PadReader
class's Read
or ReadLatest
member functions to get input from the digital buttons and Circle Pad as an nn::hid::PadStatus
structure. The Read
function can get sampling results in order of the latest result, but it cannot reacquire sampling results. When the function is called at a cycle faster than the sampling cycle, it cannot get sampling results. On the other hand, the ReadLatest
function can get only the latest sampling result. It can also reacquire sampling results, so even when it is called at a cycle faster than the sampling cycle, it can still get sampling results.
The nn::hid::PadStatus
structure's hold
member variable records the button being held when input was sampled, the trigger
variable records the button that was pressed when input was sampled, and release
variable records the button that was released when input was sampled, all mapped to bit values. For the ReadLatest
function, both the trigger
and release
members are evaluated for their states at that time, so any changes between calls are not necessarily reflected. Input bits for +Control Pad input may in fact be input from the Circle Pad emulating the +Control Pad. The actual Circle Pad input is recorded to the stick
member variable as a biaxial coordinate value.
Definition |
Corresponding Button |
---|---|
|
+Control Pad (Up) |
|
+Control Pad (Down) |
|
+Control Pad (Left) |
|
+Control Pad (Right) |
|
A Button |
|
B Button |
|
X Button |
|
Y Button |
|
L Button |
|
R Button |
|
START or SELECT (for standard operations) |
|
SELECT (only for debugging) |
|
Emulated +Control Pad (Up) input using Circle Pad |
|
Emulated +Control Pad (Down) input using Circle Pad |
|
Emulated +Control Pad (Left) input using Circle Pad |
|
Emulated +Control Pad (Right) input using Circle Pad |
Use the nn::hid::EnableSelectButton
and nn::hid::DisableSelectButton
functions to enable and disable SELECT sampling.
bool nn::hid::EnableSelectButton(); void nn::hid::DisableSelectButton();
Calling the EnableSelectButton
function returns true
when the function has successfully enabled sampling of SELECT. The function always returns false
and the sampling cannot be enabled unless the system is set to debug mode. Calling the function when SELECT sampling is already enabled returns BUTTON_SELECT_FOR_DEBUGGING
, so you can distinguish the SELECT input from the START input.
The Circle Pad may return a coordinate value even when the user is not touching it. Consequently, be sure to set an appropriate play value and clamp mode with the nn::hid::PadReader
class's SetStickClamp
and SetStickClampMode
member functions, respectively. There are three clamp modes available: circular (STICK_CLAMP_MODE_CIRCLE
), cruciform (STICK_CLAMP_MODE_CROSS
), and minimal (STICK_CLAMP_MODE_MINIMUM
). Use the GetStickClampMode
function to get the current clamp mode, and use the GetStickClamp
function to get the clamp value. Circular clamping is the default clamp setting. The following examples show clamped Circle Pad input, with the original input value coordinates given as (x, y), the distance from the origin as d, the clamp minimum and maximum values as min
and max
, the clamped coordinates as (x', y'), and the clamped distance from the origin as d'.
Circular Clamping
d <= min
(x', y') = (0, 0)
min < d < max
(x', y') = ((d - min) / d * x, (d - min) / d * y)
d > max
(x', y') = ((max - min) / d * x, (max - min) / d * y)
Cruciform Clamping
x < 0
(x', y') = (x + min, y) But x + min must be 0 or greater.
x >= 0
(x', y') = (x - min, y) But x - min must be 0 or less.
y < 0
(x', y') = (x, y + min) But y + min must be 0 or greater.
y >= 0
(x', y') = (x, y - min) But y - min must be 0 or less.
d' > (max - min)
(x', y') = ((max - min) / d * x, (max - min) / d * y)
Minimal Clamping
Minimal clamping combines circular and cruciform clamping. When using minimal clamping (minimal values), unclamped coordinate values are those that lie within the unclamped ranges of both the circular and cruciform clamping ranges. Coordinates outside this region are clamped to within this minimum.
Figure 6-1 shows how the ranges for clamped Circle Pad input coordinates change.
In STICK_CLAMP_MODE_CROSS
, for coordinates close to either the x-axis or y-axis, only the value for that axis changes (that is, if close to the x-axis, only the x-coordinate value changes).
Clamped input coordinate values for either axis produce output values as shown below in any of the clamp modes.
In any of the clamp modes, the output value is 0
until the input value passes the min
threshold, and the output value is ±(max - min)
after the input value passes the max
threshold, with the clamped output value ranging from 0
to ±(max - min)
.
In any of the clamp modes, the obtainable output coordinates describe a circle with radius (max – min)
. Choose the appropriate clamp mode for the input you need. STICK_CLAMP_MODE_CIRCLE
preserves the angle of the input coordinates from the origin and is better suited to uses requiring finer directional control. STICK_CLAMP_MODE_CROSS
only outputs the coordinate value for the closest axis, either the x-axis or the y-axis, and is better suited to uses emphasizing axial directional input, such as when selecting items from a menu.
Normalizing Circle Pad Coordinate Values
The nn::hid::PadReader
class's NormalizeStick
member function takes the Circle Pad’s coordinate value obtained from the Read
or ReadLatest
member functions and normalizes it to an f32
floating-point value between –1.0 and +1.0 with (max – min) = 1.0
.
Use the NormalizeStickWithScale
function to normalize Circle Pad coordinate values with a sensitivity adjustment. Configure the sensitivity adjustment by calling the SetNormalizeStickScaleSetting
and specifying values for the scale
(default of 1.5
) and threshold
(default of 141
) parameters. Normalizing multiplies values that are lower than threshold
by (1/scale
), while values equal to threshold
or higher gradually approach ±1.0
. Use this feature to make the Circle Pad respond as if its movable range were scale
times big.
Emulating the +Control Pad on the Circle Pad
The +Control Pad emulation bits for Circle Pad input (BUTTON_EMULATION_*
) are used to judge which direction is being indicated by the controller, by using circular clamping with a lower limit of 40 and an upper limit of 145. The values set using the SetStickClamp
and SetStickClampMode
functions are not used for this purpose. Positive and negative values for the x-axis and y-axis are interpreted to mean left, right, up, and down, respectively, and the bit for a particular direction is set if that direction is within 120° of the input direction. Consequently, the 30° overlap zone between directions is interpreted as a diagonal input.
6.2.1.1. Circle Pad Hardware Performance
The Circle Pad included in the 3DS system is made up of a resistive plate, shaft, and key top. This composite structure includes some tolerance, resulting in a slight lag in response compared to the analog sticks for the Nintendo GameCube or Wii consoles when reversing the direction of input.
Actual input values might also differ even when moving the Circle Pad along the same path of movement. This phenomenon is called path dependence and is also caused due to the Circle Pad hardware.
For example, when moving the Circle Pad from the left side to the top, going past the center, the shaft is touching the left edge of the key top, with input values tending more toward -X
. Conversely, when moving the Circle Pad from the right side to the top, going past the center, the shaft is touching the right edge of the key top, with input values tending more toward +X. Input values exhibit this same tendency in the vertical direction for ±Y
values.
We recommend that your application factor in some tolerance for this phenomenon, such as for input detection thresholds.
6.2.2. Touch Panel
Call the nn::hid::TouchPanelReader
class's Read
or ReadLatest
member functions to get input from the touch panel as an nn::hid::TouchPanelStatus
structure. The Read
function can get sampling results in order of the latest result, but it cannot reacquire sampling results. When the function is called at a cycle faster than the sampling cycle, it cannot get sampling results. On the other hand, the ReadLatest
function can get only the latest sampling result. It can also reacquire sampling results, so that even when it is called at a cycle faster than the sampling cycle, it can still get sampling results.
The nn::hid::TouchPanelStatus
structure's x
and y
members record the touch panel input coordinates in pixels, with the upper-left corner as the origin (when holding the system so that the lower screen is toward the user). Note that the coordinate axes on the touch panel differ from the LCD coordinate axes used in the GX library. Areas in the five-dot region on the outermost of the screen that is difficult to touch are returned as clamped coordinate values. Accordingly, the x values actually returned are between 5 and 314, and the y values between 5 and 234.
x = nn::gx::DISPLAY1_WIDTH - touchPanel.y; y = nn::gx::DISPLAY1_HEIGHT - touchPanel.x;
The touch
member records whether the stylus is touching the touch panel. The member value is 0
if the stylus is not touching the panel and 1
if it is.
6.2.3. Accelerometer
Call the Read
or ReadLatest
member functions of the nn::hid::AccelerometerReader
class to get input from the accelerometer as an nn::hid::AccelerometerStatus
structure. The Read
function can get sampling results in order of the latest result, but it cannot reacquire sampling results. When the function is called at a cycle faster than the sampling cycle, it cannot get sampling results. On the other hand, the ReadLatest
function can get only the latest sampling result. It can also reacquire sampling results, so that even when it is called at a cycle faster than the sampling cycle, it can still get sampling results.
The accelerometer is turned on when an instance of the AccelerometerReader
class is created and it is turned off when an instance is destroyed. To conserve battery life, only instantiate the AccelerometerReader
class when it is used. However, if you repeatedly create and destroy an instance of the AccelerometerReader
class in a function that is called every frame (for example), the frequent power cycling wastes battery power and leads to unexpected malfunctions.
The x
, y
, and z
members of the nn::hid::AccelerometerStatus
structure record input from the accelerometer's three axes. The x-axis is equivalent to the left-right axis on the +Control Pad, the y-axis is equivalent to a line perpendicular to the lower LCD screen, and the z-axis is equivalent to the up-down axis on the +Control Pad. The raw input values from these three axes do not accurately describe the acceleration of the system, so be sure to first convert the raw values to G units using the nn::hid::AccelerometerReader
class's ConvertToAcceleration
member function before using them in your application.
The accelerometer’s zero point may vary by up to 0.05 G over time and 0.08 G due to temperature changes, for a maximum variance of 0.13 G. Applications that use the accelerometer must account for this variance. Variance over time can be mostly rectified by calibrating, but variance due to temperature can reoccur in a short period even after calibrating. This 0.08 maximum temperature-induced variance equates to a roughly five-degree tilt. When detecting movements of finer precision, we recommend calibrating immediately if the user notices anything strange.
Hold the Y or B Button for three seconds while the HOME Menu is displayed to calibrate the accelerometer.
The accelerometer’s maximum sensitivity may also vary by up to ±8%. After factoring in the zero-point variance offset, the maximum input value is 10% less than the theoretical maximum value of approximately 1.8 G, at 1.62 G. Applications must not depend on any input values greater than this.
The output value of the accelerometer has a superimposed at-rest noise of ±0.02 G. The output value has an additional superimposed noise of ±0.05 G from conductive vibration when the speaker is outputting sound. The effect of this noise input must be considered when detecting small acceleration.
The conductive vibration is highest in the 1-kHz range. The effect can be reduced by adjusting the sound playback volume.
Call the SetSensitivity
member function of the nn::hid::AccelerometerReader
class to set the input play and detection sensitivity values, and use the GetSensitivity
member function to get these values. Either of these values may be set in the range from 0 to MAX_OF_ACCELEROMETER_SENSITIVITY
, with the default values of 0 for the play and MAX_OF_ACCELEROMETER_SENSITIVITY
for the detection sensitivity. Setting the detection sensitivity to 0 means only 0 will be returned. Specifying MAX_OF_ACCELEROMETER_SENSITIVITY
for this setting gives a response based directly on the value read from the device.
Looking at the accelerometer input value as a set of triaxial coordinates, the accelerometer change value for any axis is interpreted to be within the play range if it is no greater than the play value. Values within the play range are output as 0, and changes within this range are output as no change at all.
The following figure shows the relationship between the accelerometer input and output values, and the play
range. Use this play range setting to minimize any false readings caused by small changes in position, such as being bumped by the user's hands.
Use the nn::hid::AccelerometerReader
class to carry out axial rotation on output values (already adjusted for sensitivity and play) after adjusting for an offset. You can also use axial rotation of output values to apply any angle tilt to the accelerometer.
class nn::hid::AccelerometerReader { void EnableOffset(); void DisableOffset(); bool IsEnableOffset() const; void SetOffset(s16 x, s16 y,s16 z); void GetOffset(s16* pX, s16* pY, s16* pZ) const; void ResetOffset(); }
Enable offsetting of output values by calling EnableOffset
, and disable by calling DisableOffset
. Call IsEnableOffset
to get the current setting; a return value of true
indicates that the offset is enabled. When the offset is enabled, each component of the offset value is subtracted from each component of the output value.
Set the offset value itself by calling SetOffset
. Use SetOffset
to specify offset values individually for each axis. Note that each component of the offset value is specified not as an acceleration value (in G), but rather as a pre-conversion sampling value. Use the GetOffset
function to get the current setting. Call ResetOffset
to return the offset to the initial value of 0 for each component.
The SetOffsetFromBaseStatus
function has been removed from CTR-SDK.
Do not use the offset value settings to calibrate the accelerometer.
class nn::hid::AccelerometerReader { void EnableAxisRotation(); void DisableAxisRotation(); bool IsEnableAxisRotation() const; void SetAxisRotationMatrix(const nn::math::MTX34 mtx); void GetAxisRotationMatrix(nn::math::MTX34* pMtx) const; void ResetAxisRotationMatrix(); }
Enable axial rotation of output values by calling EnableAxisRotation
, and call DisableAxisRotation
to disable. Rotation is disabled by default. Call IsEnableAxisRotation
to get the current setting; a return value of true
indicates that rotation is enabled. When enabled, the previously set rotation matrix is applied to the output value.
To set the rotation matrix (a 3 × 4 matrix), call SetAxisRotationMatrix
; to get the currently set matrix, call GetAxisRotationMatrix
. Call ResetAxisRotationMatrix
to revert to the identity matrix (no rotation).
When both an offset rotation and axial rotation are enabled, the offset is applied first, and then the rotation is applied to the resulting value.
Do not use the axial rotation settings to calibrate the accelerometer.
6.2.3.1. Cautions for Implementing Application-Specific Calibration Routines
Applications are now prohibited from implementing their own calibration routines.
6.2.4. Gyro Sensor
Call the nn::hid::GyroscopeReader
class’s Read
or ReadLatest
member functions to get the gyro sensor’s input value as a nn::hid::GyroscopeStatus
structure. The Read
function can get sampling results in order of the latest result, but it cannot reacquire sampling results. When the function is called at a cycle faster than the sampling cycle, it cannot get sampling results. On the other hand, the ReadLatest
function can get only the latest sampling result. It can also reacquire sampling results, so even when it is called at a cycle faster than the sampling cycle, it can still get sampling results.
The gyro sensor is turned on when an instance of the GyroscopeReader
class is created, and it is turned off when an instance is destroyed. To conserve battery life, only instantiate the GyroscopeReader
class when it is used. However, if you repeatedly create and destroy an instance of the GyroscopeReader
class in a function that is called every frame (for example), the frequent power cycling wastes battery power and leads to unexpected malfunctions.
The speed
, angle
, and direction
members of the nn::hid::GyroscopeStatus
structure record the gyro sensor’s angular velocity, angle of rotation, and 3D attitude. The angular velocity and angle of rotation values are 3D vectors, with the x
component recording the pitch direction, the y
component the yaw direction, and the z
component the roll direction. An angular velocity of 360 dps is represented by a value of 1.0
, and an angle of rotation of 360° by a value of 1.0
. Pitch is the degree of tilt forward and back about the axis running along the width of the lower screen, yaw is rotation about an axis perpendicular to the lower screen, and roll is tilt to left and right about the axis running along the height of the lower screen.
The gyro sensor’s maximum sensitivity to angular velocity can vary by up to ±8%. After factoring in the zero-point variance offset, the maximum input value is 10% less than the theoretical maximum value of approximately 1800 dps, at 1620 dps. Applications must not depend on any input values greater than this.
The 3D attitude is recorded as a 3 × 3 matrix. This matrix is calculated based on the angular velocity, and you can change the angular velocity multiple referenced in calculation by calling the nn::hid::GyroscopeReader::SetDirectionMagnification
function. Specify a multiple of 1.0 to use the output angular velocity without modification in calculation. Specify 2.0 to use a value of double the detected angular velocity.
The nn::hid::GyroscopeReader
class detects when the system is at rest and automatically calibrates the zero-point offset (zero-point drift correction). The correction setting is applied alike to all directions.
class nn::hid::GyroscopeReader { void EnableZeroDrift(); void DisableZeroDrift(); bool IsEnableZeroDrift() const; f32 GetZeroDriftEffect() const; void ResetZeroDriftMode(); void SetZeroDriftMode(const nn::hid::ZeroDriftMode& mode); void GetZeroDriftMode(nn::hid::ZeroDriftMode& mode) const; }
To enable zero-point drift correction, call EnableZeroDrift
; to disable it, call DisableZeroDrift
. Correction is enabled by default. Call IsEnableZeroDrift
to get the current setting; a return value of true
indicates that drift correction is enabled. When enabled, the previously set rotation matrix is applied to the output value. The return value from a call to GetZeroDriftEffect
indicates the degree of drift correction being applied. A value of 0 indicates that no correction has been performed (movement was detected). Values approaching 1.0 indicate stability (no movement).
The degree of correction and the angular velocity at which the system is considered at rest both depend on the drift correction mode. Set and get the zero-point drift correction mode using the SetZeroDriftMode
and GetZeroDriftMode
functions, and use ResetZeroDriftMode
to revert to the initial value (GYROSCOPE_ZERODRIFT_STANDARD
).
Definition |
Description |
---|---|
|
Correction is applied more loosely, and constant-velocity movement might not be detected. |
|
Standard correction. (Default.) |
|
Correction is applied strictly, allowing the detection of more precise movements. |
The zero-point offset may vary by up to 50 dps when zero-point drift correction is disabled. An application must adequately account for this variance when disabling correction, such as when detecting extremely small movements.
Unless GYROSCOPE_ZERODRIFT_LOOSE
is specified, the correction may not work as well even when the system is at rest. We recommend implementing your application to account for this, such as by applying GYROSCOPE_ZERODRIFT_LOOSE
correction for scenes where the user is not expected to move the system, and then returning to the previously used correction mode during other use.
Set the zero-point play correction to avoid reacting to small changes in angular velocity. The play correction setting is applied equally to all directions.
class nn::hid::GyroscopeReader { void EnableZeroPlay(); void DisableZeroPlay(); bool IsEnableZeroPlay() const; f32 GetZeroPlayEffect() const; void SetZeroPlayParam(f32 radius); void GetZeroPlayParam(f32& radius) const; void ResetZeroPlayParam(); }
Call the EnableZeroPlay
function to enable zero-point play correction and call DisableZeroPlay
to disable it. Correction is disabled by default. Call IsEnableZeroPlay
to get the current setting; a return value of true
indicates that play correction is enabled. The return value from a call to GetZeroPlayEffect
indicates the degree of play correction being applied. The return value is negative when correction is disabled and 0 or greater when correction is enabled. As the angular velocity approaches the currently set play value, the return value approaches 0. A return value of exactly 0 indicates no correction.
To set the absolute value of the angular velocity for which play correction is 0, call the SetZeroPlayParam
function; to get the current setting, call GetZeroPlayParam
. Call ResetZeroPlayParam
to revert to the initial value of 0.005. An absolute angular velocity of 360 dps is represented by a value of 1.0.
If both zero-point drift and play correction are enabled, values are corrected first for drift and then for play.
Use the nn::hid::GyroscopeReader
class to apply axial rotation to angular velocity after correcting for zero-point drift and play. The class calculates the rotation matrix based on output values when the system is at rest, allowing you to implement gyro sensor calibration in your application. You can also use axial rotation of output values to apply any angle tilt to the gyro sensor.
class nn::hid::GyroscopeReader { void EnableAxisRotation(); void DisableAxisRotation(); bool IsEnableAxisRotation() const; void SetAxisRotationMatrix(const nn::math::MTX34 mtx); void GetAxisRotationMatrix(nn::math::MTX34* pMtx) const; void ResetAxisRotationMatrix(); }
To enable axial rotation of angular velocity, call EnableAxisRotation
; to disable, call DisableAxisRotation
. Rotation is disabled by default. Call IsEnableAxisRotation
to get the current setting; a return value of true
indicates that rotation is enabled. When enabled, the currently set rotation matrix is applied to the angular velocity.
To set the rotation matrix (a 3 × 4 matrix), call SetAxisRotationMatrix
; to get the currently set matrix, call GetAxisRotationMatrix
. Call ResetAxisRotationMatrix
to revert to the identity matrix (no rotation).
The nn::hid::GyroscopeReader
class uses the accelerometer input value to correct the gyro sensor’s 3D attitude. Anytime you use the gyro sensor, you are also using the accelerometer. When passing NULL
as the value of the pAccelerometerReader
parameter to the (overloaded) constructor, the GyroscopeReader
class internally creates and uses an instance of the nn::hid::AccelerometerReader
class that has the default settings. Pass a pointer to a configured instance of the nn::hid::AccelerometerReader
class if you want to use different values for input play and detection sensitivity.
class nn::hid::GyroscopeReader { GyroscopeReader(nn::hid::AccelerometerReader* pAccelerometerReader = NULL, nn::hid::Gyroscope& gyroscope = nn::hid::GetGyroscope()); void EnableAccRevise(); void DisableAccRevise(); bool IsEnableAccRevise() const; f32 GetAccReviseEffect() const; void SetAccReviseParam(f32 revise_pw, f32 revise_range); void GetAccReviseParam(f32& revise_pw, f32& revise_range) const; void ResetAccReviseParam(); }
To enable correction using the accelerometer, call the EnableAccRevise
function; to disable, call DisableAccRevise
. Call IsEnableAccRevise
to get the current setting; a return value of true
indicates that correction is enabled. The return value from a call to GetAccReviseEffect
indicates the degree of correction being applied. The return value is always 0 when correction is disabled and 0 or greater when correction is enabled. The return value approaches 0 as the gyroscope’s 3D attitude direction (GyroscopeStatus.direction
) approaches the accelerometer’s direction.
To set the accelerometer correction parameters (weight and enabled range), call SetAccReviseParam
; to get the current settings, call GetAccReviseParam
. Call ResetAccReviseParam
to revert to the initial settings of 0.03 for weight and 0.4 for enabled range. For the revise_pw
parameter, specify a value of 0.0
through 1.0
for the accelerometer weight. Higher values mean more severe correction. For the revise_range
parameter, specify the accelerometer range to correct within. The range is 1.0
to the ±revise_range
value. For example, specify a value of 0.4
to apply correction to acceleration values from 0.6 G through 1.4 G. The correction parameters are applied equally to all directions.
When applying angular velocity axial rotation to the gyro sensor with accelerometer correction enabled, specify the same rotation matrix for use with both the gyro sensor and accelerometer. When using an instance created with the default settings, the library makes sure internally that the same matrix is used, but if you use an instance created with any other settings, you must do this yourself. If different matrices are used, the accelerometer correction might not work properly.
6.2.5. Debug Pad (Control Pad for Debugging)
Call the nn::hid::DebugPadReader
class’s Read
or ReadLatest
functions to get input from the digital buttons and two analog sticks on the debug pad as an nn::hid::DebugPadStatus
structure. The Read
function can get sampling results in order of the latest result, but it cannot reacquire sampling results. Sampling results cannot be obtained if the function is called more frequently than the sampling frequency of 16 ms. On the other hand, the ReadLatest
function can get only the latest sampling result. It can also reacquire sampling results, so even when it is called at a cycle faster than the sampling cycle, it can still get sampling results.
The nn::hid::DebugPadStatus
structure’s hold
member records which button was held at the time of sampling, the trigger
member records which button was pressed at the time of sampling, and the release
member records which button was released at the time of sampling. For the ReadLatest
function, both the trigger
and release
members are evaluated for their states at that time, so any changes between calls are not necessarily reflected. The leftStickX
and leftStickY
members record the left analog stick’s x
and y
axes, and the rightStickX
and rightStickY
members record the right analog stick’s x
and y
axes as values ranging from –1.0 to 1.0.
Call the SetStickClampMode
function to set the analog stick clamp mode, and call GetStickClampMode
to get the current setting. This value cannot be set independently for the left and right analog sticks. You can choose either the STICK_CLAMP_MODE_CIRCLE_WITH_PLAY
or STICK_CLAMP_MODE_CIRCLE_WITHOUT_PLAY
clamp mode.
Definition |
Corresponding Button |
---|---|
|
+Control Pad (Up) |
|
+Control Pad (Down) |
|
+Control Pad (Left) |
|
+Control Pad (Right) |
|
A Button |
|
B Button |
|
X Button |
|
Y Button |
|
L Trigger |
|
R Trigger |
|
ZL Trigger |
|
ZR Trigger |
|
Plus (+) Button |
|
Minus (-) Button |
|
HOME Button |
6.2.6. Circle Pad Pro
The Circle Pad Pro is an optional peripheral device that is attached to the CTR for use. Using the Circle Pad Pro in addition to the standard input devices provided with the CTR system allows you to make use of the circle pad installed in the Circle Pad Pro (hereafter called the Right Circle Pad).
The circle pad installed by default in the CTR system is simply called the Circle Pad, and the digital buttons (the ZL and ZR Buttons). Input from input devices installed in the Circle Pad Pro is transmitted to the CTR system using infrared communication. The sampling cycle covers a range from 8 ms to 32 ms in 1-ms intervals. For a detailed description of how to perform settings, see 6.2.6.4. Sampling Start and State Acquisition.
For the process flow for using Circle Pad Pro in applications, see Appendix: Process Flows for Using the Circle Pad Pro.
For more information about the C Stick, ZL Button, and ZR Button on SNAKE and for the differences from the Circle Pad Pro, see 6.2.7. C Stick.
6.2.6.1. Hardware Internal States
The Circle Pad Pro hardware has only two states: the Active State in which communication with the CTR system is possible, and the Standby State in which no communication is performed.
Internal State |
Description |
---|---|
Active State |
State in which communication with the CTR is possible. Connection to the CTR can be made only in this state. The device enters this state as soon as a battery is inserted. |
Standby State |
No communication with the CTR is performed in this state. When not in use, battery consumption is kept to a minimum to extend battery life. The device enters this state if button input and infrared communication are not performed for a period of five minutes. The device returns to Active State by pushing one of the digital buttons on the Circle Pad Pro (ZL, ZR, or R). |
The Circle Pad Pro has no indicator to display its internal state. The device’s internal state cannot be obtained from the library.
6.2.6.2. Software Internal State
Software (library) internal states transition as shown in the following diagram.
Connection status of internal states with the Circle Pad Pro is summarized in the following table.
Internal State |
Description |
---|---|
NO_CONNECTION |
Connection with the Circle Pad Pro has not been established in this state. This is the same internal state as immediately after initialization. |
CONNECTED |
Connection with the Circle Pad Pro is established and sampling is being performed in this state. |
STOPPED |
Connection with the Circle Pad Pro, which had been established up to this point, has been interrupted due to an external cause (such as battery consumption or detachment from the CTR system). This internal state is also entered when the CTR system transitions to Sleep Mode. |
This internal state is defined by the nn::hid::ExtraPad::ConnectionState
enumerator, taking a CONNECTION_STATE_*
value.
6.2.6.3. Initialization
Before Circle Pad Pro input sampling can begin, the nn::hid::ExtraPad
class initialization function must be called.
static void nn::hid::ExtraPad::Initialize(void* workingMemory);
Specify the buffer for infrared communication in workingMemory
. The buffer must have a size of 4096 bytes and a starting address of 4096 bytes. Buffers allocated from device memory cannot be used.
After initialization completes, the nn::hid::ExtraPadReader
class can be used.
6.2.6.4. Sampling Start and State Acquisition
Use the following functions to begin sampling and get the state.
class nn::hid::ExtraPad { static nn::Result StartSampling(s32 samplingThreadPriority, s32 period); static ConnectionState GetConnectionState(); static bool IsSampling(); }
Circle Pad Pro sampling is performed by calling the nn::hid::ExtraPad::StartSampling
function. Calling the function before initialization has completed generates an error with the same value as the nn::hid::MakeResultNotInitialized
function. Processing of this function requires 50 to 200 ms when connection with the Circle Pad Pro has succeeded, and 100 ms when connection has failed.
Specify a sampling thread priority level in samplingThreadPriority
. The specifiable range is 0 to 31. The higher the priority (the closer to 0), the more stable sampling becomes.
Specify the sampling cycle (in milliseconds) in period
. The specifiable range is 8 to 32.
When sampling is started, application resources are consumed and the sampling thread is created. (The thread priority level is the value specified in samplingThreadPriority
). This thread receives sampling data from the Circle Pad Pro according to the cycle setting specified in period
, and sends approximately once every second for continuous communication with the Circle Pad Pro. The higher the value specified for the sampling cycle, the larger the processing volume per unit of time. The processing burden on the application core and system core grows in proportion to the length of the cycle. To lighten the load on the system, it is most effective to set a lower value for the sampling cycle.
When the load from other processes is especially high, it interferes with sampling thread operating cycles and may cause delays in input, or disconnection of the Circle Pad Pro.
Sampling cannot start unless the Circle Pad Pro is in the Active State. It is not possible to determine whether the device is in the Active State from the library. Prompt the user to enter input from one of the digital buttons (R/ZR/ZL) on the Circle Pad Pro when the StartSampling
function sends back a return value indicating that the Circle Pad Pro cannot be found (the same value as for the nn::hid::MakeResultNoConnection
function). After you have returned to the Active State, you can reconnect.
When the StartSampling
function is called, a connection is made after a disconnect has been performed once.
To perform ongoing detection of the Circle Pad Pro, a load must be set at less than a 32-ms cycle, assuming the StartSampling
function is called at a frequency of once per second. This is smaller than sampling processing.
The nn::hid::ExtraPad::GetConnectionState
function returns the Circle Pad Pro connection state as an internal state. Reconnect after performing a disconnect once, when a disconnect occurs due to an external cause and CONNECTION_STATE_STOPPED
is returned.
The nn::hid::ExtraPad::IsSampling
function returns whether sampling is in progress. If the value true
is returned, sampling is being performed and Circle Pad Pro input is reflected in the nn::hid::ExtraPadStatus
structure acquired by the nn::hid::ExtraPadReader
class. When the value true
is returned, it guarantees that the internal state is also in a connected state (CONNECTED), but note that if a connection or disconnect is underway, the value false
could be returned even if the internal state is in a connected state.
After Circle Pad Pro sampling begins, the nn::hid::PadReader
class can no longer be used, but even if the Circle Pad Pro is not connected, input from the system’s digital buttons and circle pad is reflected in the nn::hid::ExtraPadStatus
structure obtained by the nn::hid::ExtraPadReader
class.
It is unnecessary for the application supporting the Circle Pad Pro to use the connection state to switch between classes and structures, because it uses the nn::hid::ExtraPadReader
class and the nn::hid::ExtraPadStatus
structure.
6.2.6.5. Stop Sampling
Call the nn::hid::ExtraPad::StopSampling
function to stop sampling.
static nn::Result nn::hid::ExtraPad::StopSampling();
After a disconnect has succeeded, the sampling thread is destroyed and the internal state transitions to a not connected state (NO_CONNECTION
). This function always succeeds in processing as long as initialization has already been completed when it is called.
6.2.6.6. Getting Sampling Results
You can get sampling results with the Read
or ReadLatest
function under the nn::hid::ExtraPadReader
class, reflecting the nn:: hid::ExtraPadStatus
structure. The Read
function can get sampling results in order of the latest result, but it cannot reacquire sampling results. When the function is called at a cycle faster than the sampling cycle, it cannot get sampling results. On the other hand, the ReadLatest
function can get only the latest sampling result. It can also reacquire sampling results, so even when it is called at a cycle faster than the sampling cycle, it can still get sampling results.
struct nn::hid::ExtraPadStatus { AnalogStickStatus stick; AnalogStickStatus extraStick; bit32 hold; bit32 trigger; bit32 release; u8 batteryLevel; bool isConnected; NN_PADDING2; };
The Circle Pad Pro’s input from the system is reflected in stick
, in the same way as in the nn::hid::PadStatus
structure.
Input from the Circle Pad Pro’s Right Circle Pad is reflected in extraStick
.
The hold
, trigger
, and release
member variables are the same as those in the nn::hid::PadStatus
structure. The following buttons have been added to the device.
Definition |
Corresponding Button |
---|---|
|
Circle Pad Pro ZL button |
|
Circle Pad Pro ZR button |
|
Right Circle Pad + button for upward emulation input |
|
Right Circle Pad + button for downward emulation input |
|
Right Circle Pad + button for left emulation input |
|
Right Circle Pad + button for right emulation input |
When the Circle Pad Pro is attached, it is difficult to press the R Button, so the Circle Pad Pro has been fitted with its own R Button. The application cannot detect whether the pressed R Button is the one on the CTR system or on the Circle Pad Pro.
When the Circle Pad Pro is not performing sampling, input from the CTR system is reflected in the nn::hid::ExtraPadStatus
structure in 4-ms sampling cycles by the nn::hid::ExtraPadReader
class. However, when Circle Pad Pro sampling is being performed, the sampling cycle for input from the CTR system uses the sampling cycle specified by the nn::hid::ExtraPad::StartSampling
function (between 8 and 32 ms).
Circle Pad Pro battery level is stored in two values (0 and 1) in batteryLevel
. Even when the value 0 is returned, there is approximately one day remaining of continuous operation.
Even if the value 1 is returned for remaining battery level, inserting a battery whose battery life is already partially consumed may result in the Circle Pad Pro failing to restart. If communication no longer works after inserting the battery, replace it with a new battery. This phenomenon occurs only when inserting a battery. After a new battery has been inserted, normal operation continues until the remaining battery life reaches 0.
Results of library searches for whether sampling is in progress are stored in isConnected
.
The time required until the application can get Circle Pad Pro input (input delay) is approximately 2 ms or more (2 ms + sampling cycle) when the volume of other processing being performed by the application is sufficiently small. It takes 0 sampling cycles or more to get user input from the Circle Pad Pro, plus communication time and CTR processing, which requires approximately 2 ms.
6.2.6.7. Circle Pad Clamp Processing
Functions used by the CTR system for circle pad clamp processing are defined with the same arguments as nn::hid::PadReader
class member functions. Operation is also the same as with this class.
Functions used in Right Circle Pad clamp processing use the same settings as the circle pad on the CTR system side, except that the function name Stick
is used instead of ExtraStick
. These settings can be performed individually.
class nn::hid::ExtraPadReader { void SetExtraStickClamp(s16 min, s16 max); void GetExtraStickClamp(s16* pMin, s16* pMax) const; StickClampMode GetExtraStickClampMode() const; void SetExtraStickClampMode(StickClampMode mode); f32 NormalizeExtraStick(s16 x); void NormalizeExtraStickWithScale( f32* normalized_x, f32* normalized_y, s16 x, s16 y); void SetNormalizeExtraStickScaleSettings(f32 scale, s16 threshold); void GetNormalizeExtraStickScaleSettings(f32* scale, s16* threshold) const; }
6.2.6.8. Event Notification
You can register events (nn::os::LightEvent
class) that notify changes in the connection state and completion of sampling.
class nn::hid::ExtraPad { static void RegisterConnectionEvent(nn::os::LightEvent* pLightEvent); static void UnregisterConnectionEvent(); static void RegisterSamplingEvent(nn::os::LightEvent* pLightEvent); static void UnregisterSamplingEvent(); }
Event notifications associated with changes in connection state are registered and unregistered with the *ConnectionEvent
functions, and event notifications for completing sampling are registered and unregistered with the *SamplingEvent
functions. After an event is registered, thread processing for sampling increases just slightly.
The processing load during registration of events that notify of changes in the connection state is so minor that it can be ignored. However, registration of the sampling complete event requires processing of each sampling cycle, so there is more processing load on the CPU. We recommend that these functions only be used on applications where event notification is used.
Changes in connection state can also be determined by nn::hid::ExtraPad
class member functions, GetConnectionState
and IsSampling
, and by the isConnected
structure member variable of nn::hid::ExtraPadStatus
.
6.2.6.9. Calibration Applet
A library applet called the Circle Pad Pro calibration applet is provided to calibrate the feel of controls on the Right Circle Pad within the Circle Pad Pro. Follow the guidelines that provide examples where applications supporting the Circle Pad Pro can call up the Circle Pad Pro calibration applet.
For more information, see 12.1.7. Circle Pad Pro Calibration Applet.
6.2.6.10. Differences Between the Circle Pad and the Right Circle Pad
Input coordinates sampled from the Circle Pad and the Right Circle Pad have a tendency to change as a result of electrical fluctuations, even when operations are performed intentionally using the pad directly. The amount of change in values when key positions on the pads are fixed and the probability that this might occur differs for the Circle Pad and the Right Circle Pad, as indicated in the following table. The Right Circle Pad is especially susceptible to these changes, so caution is required.
Amount of Change |
Circle Pad |
Circle Pad |
Right Circle Pad |
Right Circle Pad |
---|---|---|---|---|
–3 |
0.000006 % |
0.000006 % |
0.002 % |
0.001 % |
–2 |
0.000178 % |
0.000178 % |
0.349 % |
0.352 % |
–1 |
0.003541 % |
0.003541 % |
3.485 % |
3.273 % |
±0 |
99.949837 % |
99.949837 % |
92.326 % |
92.743 % |
+1 |
0.003541 % |
0.003541 % |
3.491 % |
3.285 % |
+2 |
0.000178 % |
0.000178 % |
0.346 % |
0.345 % |
+3 |
0.000006 % |
0.000006 % |
0.002 % |
0.001 % |
The probability of change occurring within the unit of time noted is determined according to the following formula:
Amount of sampling data obtained during unit of time × Probability from Table 6-8.
If the pad is positioned in the center after the user’s finger is removed, the value will not change from the 0 position. This is because even with some variation in value, the minimum clamping value is never exceeded.
6.2.7. C Stick
The library lets you use the C Stick on SNAKE without differentiating from the Right Circle Pad on the Circle Pad Pro, but the actual hardware is different. You must consider hardware differences when using this approach.
6.2.7.1. Properties of C Stick Hardware
The C Stick on SNAKE is an analog input device that detects minor deformations (strain) when pressure is applied to a resin stick and reads the magnitude of the deformation as changes in voltage.
Note the following points because the output value from the C Stick depends on the pressure on the stick part.
- It may be difficult to maintain the maximum input value depending on the strength of the user.
- When too much pressure is applied, the output value from the C Stick does not change (the output value is saturated).
- When the C Stick is strongly pressed from directly above, the output value from the C Stick is undefined.
- When the X Button is strongly held down, the C Stick output value might fluctuate due to the close physical proximity to the X Button.
In addition, due to variances in the performance of the sensors on each system, note the following.
- Sensitivity variances between individual systems have an effect on resolution.
Effect on the Pressure Required to Input the Maximum Value
The maximum detectable pressure range of the C Stick is approximately 240 gf. Consequently, it might be difficult for some users to apply the maximum pressure continuously.
The minimum pressure value differs depending on the clamping method and input direction. The detectable pressure range of the C Stick is summarized in the following table.
Clamping Method | Minimum Detectable Pressure | Maximum Detectable Pressure | |
---|---|---|---|
Up, Down, Left, Right (4 directions) | Diagonal 45° directions | ||
Circular (min=40, max=145) | Approx. 68 gf | Approx. 72 gf | Approx. 240 gf |
Cross (min=36, max=145) | Approx. 63 gf | Approx. 90 gf | |
Minimal (min=40, max=145) | Approx. 63 gf | Approx. 72 gf |
Effect of Saturated Output Value
When excessive force is applied to the C Stick, the output value of the C Stick no longer changes, even if more pressure is applied. This condition is called "saturation of the output value."
C Stick input is measured on two independent axes (the x-axis and y-axis). In a situation where the output value is saturated for both axes, the system can no longer detect any C Stick operations by the user.
For example, if the C Stick is moved in a large circular motion with enough force and the C Stick input axes are aligned with the system, the output value does not change for some regions at ±45° angles from the input axes. This is illustrated below.
The orange x-axis and y-axis show the orientation of the system, and the black x-axis and y-axis show the input axis of the C Stick. The orange circle shows the input pressure, the light shaded area (in the middle) shows where a raw output value can be obtained from the C Stick device, and the dark shaded area shows where the output value does not change for either axis. The area where the value does not change is drawn larger than the actual area for illustrative purposes.
The C Stick input axes are rotated 45° from the orientation of the system to minimize problems from the user's perspective.
The rotation of the input axes is handled by the system. Applications do not need to apply any rotation to C Stick input retrieved from the library.
Behavior When Strongly Pressed From Above
This action causes the stick part to bend, which may result in the following symptoms.
- The C Stick output value fluctuates even though no directional force is applied.
- The input is processed as a different direction than what the user intended.
Effect of Holding Down the X Button
When strong pressure is applied to the X Button, the C Stick output value might fluctuate even if it is not being used. This issue occurs due to the deformation of the housing when the X Button is strongly pressed, which can also cause the C Stick to bend.
This issue does not occur when the X Button is rapidly pressed, and requires significant force to occur when holding down the X Button. More force is required to reproduce this issue on the development hardware and the new Nintendo 3DS than on the new Nintendo 3DS XL.
In practice, Nintendo expects the effect of this issue on the C Stick output value to stay within the following ranges.
Clamping Method | x-axis | y-axis |
---|---|---|
Circular (min=40, max=145) | 0 to 10 | 0 to –35 |
Cross (min=36, max=145) | 0 | 0 to –38 |
Minimal (min=40, max=145) | 0 to 10 | 0 to –37 |
Effect of Sensitivity Variations Between Individual Systems on Resolution
The C Stick has slight differences in sensitivity between individual systems and is calibrated during the manufacturing process to mitigate these differences. Due to the calibration process, some systems have higher resolution than others.
Depending on the individual system and the clamping mode, it may not be possible to get all of the values within the clamping range from the C Stick. The change in value is not guaranteed to be consecutive on systems with higher resolution, and this tendency becomes even more pronounced on systems with lower resolution. Output is guaranteed, however, for the center point and the outer edge.
For this reason, Nintendo does not recommend implementations that require input from a narrow range or expect a smooth change in value.
The figure below shows the output value distribution on a low-resolution system with circular clamping applied, in addition to an example of the narrow detection range that is not recommended. The orange background represents the range of values available to the application, and the black points represent the actual values that are output. Note that the values of adjacent black dots are not guaranteed to be consecutive. The green circle on the left side shows a narrow detection range when the C Stick output value is converted to a vector length. The green square on the right shows a narrow detection range when using the raw C Stick output value. In other words, depending on the input direction, there might not be any points that satisfy the required vector length, and there might not be any points at the raw value being detected.
6.2.7.2. Differences From the Circle Pad Pro
When using input to the SNAKE C Stick with the HID library, note the following differences from the specifications and behavior of the Circle Pad Pro.
- The configurable sampling frequency is different.
- The device is never disconnected due to external factors.
- The behavior when waking from Sleep Mode is different.
- The center point and sensitivity are not calibrated in the Circle Pad Pro Calibration Applet.
Different Configurable Sampling Frequency
A sampling frequency of 10 to 21 ms can be set for the SNAKE C Stick, ZL Button, and ZR Button hardware.
The nn::hid::ExtraPad::StartSampling
function lets you specify a frequency in the range of 8 to 32 ms, but the hardware handles values below 10 ms as 10 ms, and values above 21 ms as 21 ms. The function returns data at that frequency.
We recommend setting a sampling frequency of 10 to 21 ms based on the sampling frequency of the hardware.
No Disconnects due to External Factors
The Circle Pad Pro, which was connected using infrared communication, could be disconnected by external factors such as being detached or a dead battery. The SNAKE C Stick, ZL Button, and ZR Button, however, are part of the system itself, so nn::hid::ExtraPad::GetConnectionState
never returns nn::hid::ExtraPad::CONNECTION_STATE_STOPPED
due to external factors.
Because the battery never runs out, the batteryLevel
member of the nn::hid::ExtraPadStatus
structure is always set to 1 while input is being sampled.
Different Behavior When Waking From Sleep Mode
The Circle Pad Pro and the C Stick have the following differences in behavior when the system enters Sleep Mode without calling nn::hid::ExtraPad::StopSampling
.
- Circle Pad Pro
Infrared communication stops when the system enters Sleep Mode, which would cause sampling requests to fail immediately after the system wakes. To prevent this, the sampling group is removed and the library treats the Circle Pad Pro as having been stopped. However, because the internal state changes based on the infrared communication status, the internal state before detecting the stoppage is unstable.
Specifically, thenn::hid::ExtraPad::GetConnectionState
function generally returnsCONNECTION_STATE_STOPPED
, but it might sometimes returnCONNECTION_STATE_CONNECTED
. - C Stick
Sampling requests succeed immediately after waking from Sleep Mode and the connection state is retained.
Specifically, thenn::hid::ExtraPad::GetConnectionState
function always returnsCONNECTION_STATE_CONNECTED
.
The value returned by the nn::hid::ExtraPad::IsSampling
function also depends on the internal state, so that value changes at the same time.
To handle this behavior, we recommend calling nn::hid::ExtraPad::StopSampling
to stop the sampling process before entering Sleep Mode. If you call nn::hid::ExtraPad::StopSampling
, resume sampling by calling nn::hid::ExtraPad::StartSampling
after waking from Sleep Mode. Nintendo also recommends this procedure before and after calling functions where the system could enter Sleep Mode before control returns to the application, such as the HOME Menu or a library applet.
No Center Point or Sensitivity Calibration in Circle Pad Pro Calibration Applet
SNAKE works differently from CTR when the Circle Pad Pro Calibration Applet is called. On SNAKE, the system enters a mode to check the operation of the C Stick and displays a description of how the C Stick is calibrated.
This is because the C Stick center point is automatically calibrated at the following times.
- When power is turned on
If the C Stick is pressed when the power is turned on, it will not be calibrated correctly. Even so, it is still possible to correctly calibrate the C Stick after waking from Sleep Mode, as described below. - When waking from Sleep Mode
It is physically impossible to operate the C Stick while the system is closed, so no C Stick input ever occurs during sleep. You can safely assume that the device was calibrated correctly in this case. - During sampling (when the user is not using the control)
The application does not need to consider the effects of this calibration because it takes place when the C Stick output value is below the lower limit of the clamping.
The center point calibration after Sleep Mode is triggered when the system wakes. Consequently, the calibration does not take place when the system is reopened if the application rejects the Sleep Mode request. The message that is displayed in the calibration applet tells the user to put the system to sleep by closing it.
Note that center point calibration does not take place when an application rejects the sleep request for one of the following reasons.
- The application does not want to disconnect even if the system is closed.
- The system does not enter Sleep Mode when closed while in Sound Mode if headphones are connected.
6.3. MIC Library
The MIC library handles audio input from the microphone obtained by automatic sampling.
Call the nn::mic::Initialize
function to initialize the MIC library. The microphone can be used after this function successfully completes, but to sample microphone input in an application, you must take care of other preparations required for sampling, such as allocating the buffers for storing the sampling results, configuring the microphone gain, and dealing with microphone power control.
6.3.1. Buffer Allocation
The application allocates the buffer for storing sampling results. The starting address of the buffer must be 4096-byte aligned, its size must be a multiple of 4096 bytes, and it must be allocated from non-device memory. Pass the allocated buffer to the MIC library using the nn::mic::SetBuffer
function. The last 4 bytes of the buffer are set aside for management purposes, so the actual space available for storing sampling results is the value specified minus these 4 bytes. Use the nn::mic::GetSamplingBufferSize
function to get the size of the buffer used to store sampling results.
Calling the nn::mic::SetBuffer
function after a buffer is already allocated causes an error. To reallocate a buffer, first call the nn::mic::ResetBuffer
function and then call the nn::mic::SetBuffer
function.
6.3.2. Microphone Gain Settings
The microphone gain setting determines the amplification multiple for audio input from the microphone.
Use the nn::mic::GetAmpGain
and nn::mic::SetAmpGain
functions to get and set the gain. You can set the gain in the range from 0 to 119 with each increase of 1 representing a 0.5 dB increment. 0 equates to 10.5 dB (a multiple of roughly 3.4), and 119 equates to 70.0 dB (a multiple of roughly 3162). As shown in the following table, the four gain levels for the NITRO microphone can be used by converting to gain setting values.
Ratio |
dB |
Gain Setting Values |
---|---|---|
20 x |
26.0 |
31 |
40 x |
32.0 |
43 |
80 x |
38.0 |
55 |
160 x |
44.0 |
67 |
The microphone gain setting is set to AMP_GAIN_DEFAULT_VALUE
when the library is initialized.
6.3.3. Microphone Power Control
Use the nn::mic::SetAmp
function to control power to the microphone (microphone amp). Pass true
as an argument to turn the microphone on. Microphone input is unstable immediately after turning the microphone on or recovering from sleep, so the first second of sampling is forcibly muted.
Call the nn::mic::GetAmp
function to get the current setting.
6.3.4. Start Sampling
The preceding steps have prepared the microphone for sampling. Call the nn::mic::StartSampling
function to start sampling automatically.
nn::Result nn::mic::StartSampling( nn::mic::SamplingType type, nn::mic::SamplingRate rate, s32 offset, size_t size, bool loop);
Specify the type of data to get in the type
parameter. You can choose from the following four types of data, depending on factors such as your bit width and sign requirements.
Type |
Bit Width |
Signed |
Sampling Value Range |
Sampling Value Indicating Mute |
---|---|---|---|---|
|
8 |
None |
0 to 255 |
128 |
|
16 |
None |
0 to 65,535 |
32,768 |
|
8 |
Yes |
–128 to 127 |
0 |
|
16 |
Yes |
–32,768 to 32,767 |
0 |
Specify the sampling rate in the rate
parameter. You can choose from the following four sampling rates.
Rate |
Sampling Rate |
---|---|
|
32.73 kHz (32,728.498046875 Hz) |
|
16.36 kHz (16,364.2490234375 Hz) |
|
10.91 kHz (10,909.4993489583 Hz) |
|
8.18 kHz (8182.12451171875 Hz) |
Specify the offset from the start of the sampling data storage location (the start of the buffer) in the offset
parameter. This value must be a multiple of 2 that is greater than or equal to 0.
Specify the number of bytes of sampling results to store for each sample in the size
parameter. This value must be small enough that (offset
+ size
) is no larger than the size specified when allocating the buffer.
Specify whether to continue sampling after storing size
bytes of sampling results in the loop
parameter. Pass true
to continue sampling and treat the buffer as a ring buffer (the region of size
bytes starting from offset
).
If you call the nn::mic::StartSampling
function while already sampling, the current sampling session is terminated and a new sampling session is started. If this function is called while the system is closed, it returns nn::mic::ResultShellClose
and sampling does not begin. If the system is closed during ongoing sampling, sampling is automatically stopped and resumes when the system is opened.
Call the nn::mic::AdjustSampling
function to change the sampling rate during sampling. When you change the sampling rate this way, the new rate is used for sampling data starting from the current storage location.
Call the nn::mic::IsSampling
function to check whether the microphone is currently being sampled. However, this entails a significant processing burden because it sends a request directly to the device. It is not suitable for operations that involve calling at each frame.
Call the nn::mic::SetLowPassFilter
function and pass true
in the enable
parameter to apply a low-pass filter using a microphone input cutoff frequency of 45% of the sampling rate. The default value is false
(no filtering). However, data sampled with a sampling rate of SAMPLING_RATE_32730
was sampled using a low-pass filter, so applying a low-pass filter using this function has no effect.
6.3.4.1. Synchronization With Sound Processing
The DSP that processes sound and the processor that handles the microphone are separate devices. It is not possible to synchronize them perfectly. It is possible, however, to keep them in near synchronization by correcting differences in timing.
Keep the timers in sync by periodically calculating mismatches between the sampling count and elapsed time of the microphone, and the playback time of the audio, and correct any offset. Also correct mismatches in the sampling frequency by calling Voice::SetPitch
.
6.3.5. Getting Sampling Results
Call the nn::mic::GetLastSamplingAddress
function to get the address of the most recent sampling results.
The nn::os::Event
class instance, which is obtainable by a call to the nn::mic::GetBufferFullEvent
function, is a manually resetting event that enters the signaled state when the buffer cannot store any more sampling results. This event enters the signaled state when the buffer has stored size
bytes of sampling results from the offset
position and remains in the signaled state until the ClearSignal
function is called.
Due to differences in individual CTR systems in terms of the range of values obtained as input, the guaranteed input value ranges for each sampling type are defined by the TYPE_*_GUARANTEED_INPUT_MIN(MAX)
constants. Do not design applications that expect values outside of these guaranteed input ranges when determining if there is microphone input.
Sampling Type |
Lower Limit |
Upper Limit |
---|---|---|
|
27 |
228 |
|
7105 |
58415 |
|
–101 |
100 |
|
–25,663 |
25,647 |
The default behavior is for microphone input values to be clamped to within these guaranteed ranges. If you need input values across a broader range, you can disable clamping by calling the nn::mic::SetClamp
function, but note that you might still be unable to obtain input values from outside the guaranteed input ranges.
6.3.5.1. Prohibited Ranges for Microphone Input Determination
Call the nn::mic::GetForbiddenArea
function, passing in the microphone amplifier gain setting and sampling type, to get the lower and upper limits for sampling results that must not be recognized as microphone input. The application must not recognize any sampling results with values between these lower and upper limits as microphone input. The obtainable values are shown in Table 6-15.
The rows shaded in light gray in Table 6-15 show where the gain is 68 (44.5 dB) or higher. In these ranges, microphone input is subject to marked static and other noise due to sources such as system speaker output, button clicks, and the sound of the stylus contacting the touch panel. These ranges are unsuitable for determining whether there is any microphone input based on amplitude levels. The rows in the table shaded in darker gray show where the gain is 104 (62.5 dB) or higher, where the full range of microphone input is noisy. Only use these ranges when invalid microphone input is not a problem.
Sampling Type |
Gain |
dB |
Prohibited Ranges for Microphone Input Detection (Noise Component) |
---|---|---|---|
|
0 to 31 |
10.5 to 26.0 |
125 to 131 |
32 to 43 |
26.5 to 32.0 |
123 to 133 |
|
44 to 55 |
32.5 to 38.0 |
119 to 137 |
|
56 to 67 |
38.5 to 44.0 |
112 to 144 |
|
68 to 80 |
44.5 to 50.5 |
96 to 160 |
|
81 to 91 |
51.0 to 56.0 |
77 to 179 |
|
92 to 103 |
56.5 to 62.0 |
18 to 238 |
|
104 to 119 |
62.5 to 70.0 |
0 to 255 |
|
|
0 to 31 |
10.5 to 26.0 |
32,000 to 33,536 |
32 to 43 |
26.5 to 32.0 |
31,488 to 34,048 |
|
44 to 55 |
32.5 to 38.0 |
30,464 to 35,072 |
|
56 to 67 |
38.5 to 44.0 |
28,672 to 36,864 |
|
68 to 80 |
44.5 to 50.5 |
24,576 to 40,960 |
|
81 to 91 |
51.0 to 56.0 |
19,712 to 45,824 |
|
92 to 103 |
56.5 to 62.0 |
4608 to 60,928 |
|
104 to 119 |
62.5 to 70.0 |
0 to 65,535 |
|
|
0 to 31 |
10.5 to 26.0 |
–3 to +3 |
32 to 43 |
26.5 to 32.0 |
–5 to +5 |
|
44 to 55 |
32.5 to 38.0 |
–9 to +9 |
|
56 to 67 |
38.5 to 44.0 |
–16 to +16 |
|
68 to 80 |
44.5 to 50.5 |
–32 to +32 |
|
81 to 91 |
51.0 to 56.0 |
–51 to +51 |
|
92 to 103 |
56.5 to 62.0 |
–110 to +110 |
|
104 to 119 |
62.5 to 70.0 |
–128 to +127 |
|
|
0 to 31 |
10.5 to 26.0 |
–768 to +768 |
32 to 43 |
26.5 to 32.0 |
–1280 to +1280 |
|
44 to 55 |
32.5 to 38.0 |
–2304 to +2304 |
|
56 to 67 |
38.5 to 44.0 |
–4096 to +4096 |
|
68 to 80 |
44.5 to 50.5 |
–8192 to +8192 |
|
81 to 91 |
51.0 to 56.0 |
–13,056 to +13,056 |
|
92 to 103 |
56.5 to 62.0 |
–28,160 to +28,160 |
|
104 to 119 |
62.5 to 70.0 |
–32,768 to +32,767 |
6.3.6. Stop Sampling
Use the nn::mic::StopSampling
function to stop sampling. This function only stops sampling. It does not turn off power to the microphone.
6.3.7. Closing the MIC Library
To stop using input from the microphone, such as when quitting an application, first stop sampling, and then do the following.
- Call the
nn::mic::SetAmp
function and passfalse
as an argument to turn off the microphone power. - Call the
nn::mic::ResetBuffer
function to free up allocated buffers. - Call the
nn::mic::Finalize
function to close the MIC library.
If you want to stop sampling for a time and then sample more later, just turn off the microphone power to save battery life.
The microphone gain setting is set to AMP_GAIN_DEFAULT_VALUE
when the library is closed.
6.4. Camera Library and Y2R Library
The camera library handles operations for the camera on the system. Images captured from the camera are only in YUV format. For conversion to RGB format, we recommend using the YUVtoRGB circuit, which also supports conversion to the native GPU format. Use the Y2R library for YUVtoRGB circuit operations.
6.4.1. Initializing
To initialize the camera library, call the nn::camera::Initialize
function; to initialize the Y2R library, call the nn::y2r::Initialize
function. However, calling the initializers is not sufficient to prepare the camera and the YUVtoRGB circuit. You must also configure the settings for each and ready them to send and receive data.
6.4.2. Photography Environment Settings
This section describes the configurable settings for the photography environment. You must specify which camera to configure by selecting from the following options.
Setting Value |
Target Camera |
---|---|
|
Specifies no cameras. Used to, for example, configure standby settings. |
|
Right outer camera. |
|
Inner camera. |
|
Right outer camera and inner camera. |
|
Left outer camera. |
|
Both left and right outer cameras. |
|
Right outer camera and inner camera. |
|
All cameras. Both left and right outer cameras and inner camera. |
Using both outer cameras together with SELECT_OUT1_OUT2
is also called stereo mode, with the images taken by the outer cameras appearing in 3D. When doing so, make sure the photographic environment settings of the two cameras are the same.
For more information about stereoscopic display and calibration methods using the stereo camera, see 3DS Programming Manual: Advanced Graphics.
Configuring the photographic environment settings while capturing could cause distortion of the captured images. Consequently, first pause capture and then configure the settings.
When specifying SELECT_ALL
, note that the cameras selected differ from in the past when there was only one outer camera.
Note that configuring the photographic environment settings while the camera is restarting may cause the library’s processing to block for an extended period.
Image data taken by the camera includes variations based on the camera itself, ambient light, and color variation in the subject. For example, camera variations include angle-of-view, resolution, color reproducibility, rotation, and distortion. An image taken with the same angle and camera setting is different in each system.
Resolution
Call the nn::camera::SetSize
function to configure the resolution of the images captured by the camera. The resolution is the size of the image before trimming. Choose from the following resolution options.
Setting Value |
Resolution |
Description |
---|---|---|
|
640 × 480 |
VGA |
|
320 × 240 |
QVGA |
|
160 × 120 |
QQVGA |
|
352 × 288 |
CIF (aspect ratio of 11:9) |
|
176 × 144 |
QCIF (aspect ratio of 11:9) |
|
256 × 192 |
Resolution of the DS LCD. |
|
512 × 384 |
Resolution at twice the height and width of the DS LCD. |
|
400 × 240 |
Resolution of the 3DS upper LCD |
|
320 × 240 |
Resolution of the 3DS lower LCD. Same as QVGA. |
Images captured using the SIZE_CIF
, SIZE_QCIF
, and SIZE_CTR_TOP_LCD
settings have a 4:3 aspect ratio, with the left and right sides of the images trimmed.
Call the nn::camera::SetDetailSize
function to configure the other resolution settings. You can freely set the resolution of the captured images to crop from the original VGA-sized image by specifying the height and width of the output image. Make sure that the width and height are greater than or equal to the output image size. If not trimming, the output image width times the height must be a multiple of 128. To maintain compatibility, be sure to specify the cropX0
parameter as both an even number and a value that results in a multiple of 4 (cropX1
- cropX0
+ 1).
To take images in stereo mode, make sure that the two cameras are set to the same resolution.
Sharpness
Call the nn::camera::SetSharpness
function to set a camera's sharpness. Valid sharpness values range from -4 to +5.
Exposure
Call the nn::camera::SetExposure
function to set the camera's exposure. Valid exposure values range from -5 to +5.
Call the nn::camera::SetAutoExposure
function to enable or disable the auto exposure (AE) feature. The exposure value is unstable immediately after starting up the camera, so we recommend leaving the AE feature enabled. Call the nn::camera::IsAutoExposure
function to get the camera's current AE setting. Note that the AE feature is enabled when setting the exposure with the nn::camera::SetExposure
function, even if you have disabled the AE feature previously.
Call the nn::camera::SetAutoExposureWindow
function to set the region used as the standard for automatic calculation when in automatic exposure mode. Specify this region as a portion of the 640 × 480 VGA maximum size of a captured image, changing the starting coordinates, width, and height within the following ranges. The three rightmost columns of the table show the initial setting values for the inner, right outer, and left outer cameras.
Parameter |
Setting |
Setting Value Range |
IN1 |
OUT1 |
OUT2 |
---|---|---|---|---|---|
|
Starting coordinate (horizontal) |
0 to 600 (in 40-pixel increments) |
80 |
0 |
0 |
|
Starting coordinate (vertical) |
0 to 450 (in 30-pixel increments) |
60 |
0 |
0 |
|
Width |
40 to 640 (in 40-pixel increments) 640 or less when summed with |
480 |
640 |
640 |
|
Height |
30 to 480 (in 30-pixel increments) 480 or less when summed with |
360 |
480 |
480 |
Frame Rate
Call the nn::camera::SetFrame Rate
function to set how many frames per second (fps) to capture. You can choose from the following values. When in stereo mode, use the same frame rate for both outer cameras.
Setting Value |
Frame Rate |
---|---|
|
Fixed at 15 fps. |
|
Automatically varies from 15 to 5 fps depending on the available light. |
|
Automatically varies from 15 to 2 fps depending on the available light. |
|
Fixed at 10 fps. |
|
Fixed at 8.5 fps. |
|
Fixed at 5 fps. |
|
Fixed at 20 fps. |
|
Automatically varies from 20 to 5 fps depending on the available light |
|
Fixed at 30 fps. |
|
Automatically varies from 30 to 5 fps depending on the available light. |
|
Automatically varies from 15 to 10 fps depending on the available light. |
|
Automatically varies from 20 to 10 fps depending on the available light. |
|
Automatically varies from 30 to 10 fps depending on the available light. |
White Balance
Call the nn::camera::SetWhiteBalance
function to set the camera's white balance. Choose from the following white balance setting values.
Setting Value |
Aliases |
Description |
---|---|---|
|
|
Automatic white balance. |
|
|
Tungsten light (incandescent light bulb). |
|
|
White fluorescent. |
|
|
Daylight. |
|
|
Cloud cover. |
|
Sunset. |
|
|
|
Shade. |
Call the nn::camera::SetWhiteBalance
function and configure the white balance setting to WHITE_BALANCE_AUTO
to enable the auto white balance feature. Any other setting value will disable the auto white balance feature.
Use the nn::camera::SetAutoWhiteBalance
function to enable or disable the auto white balance feature after enabling it by using WHITE_BALANCE_AUTO
. If some other white balance setting was used other than WHITE_BALANCE_AUTO
, you cannot use the nn::camera::SetAutoWhiteBalance
function. Use the nn::camera::IsAutoWhiteBalance
function to get the current auto white balance setting.
Call nn::camera::SetAutoWhiteBalanceWindow
to set the region used as the benchmark for auto white balance function when automatically calculating the white balance. Specify this region as a portion of the 640 × 480 VGA maximum size of a captured image, changing the starting coordinates, width, and height within the following ranges. The three rightmost columns of the table show the initial setting values for the inner, right outer, and left outer cameras.
Parameter |
Setting |
Setting Value Range |
IN1 |
OUT1 |
OUT2 |
---|---|---|---|---|---|
|
Starting coordinate (horizontal) |
0 to 600 (in 40-pixel increments) |
0 |
0 |
0 |
|
Starting coordinate (vertical) |
0 to 450 (in 30-pixel increments) |
0 |
0 |
0 |
|
Width |
40 to 640 (in 40-pixel increments) 640 or less when summed with |
640 |
640 |
640 |
|
Height |
30 to 480 (in 30-pixel increments) 480 or less when summed with |
480 |
480 |
480 |
The automatic adjustment of the white balance may be degraded if there is little change in the input images, or the input image has little contrast, as with a plain wall.
Photo Mode
Call the nn::camera::SetPhotoMode
function to set the camera's photo mode to match the subject of the photo. Choose from the following values.
Setting Value |
Photo Mode |
Description |
---|---|---|
|
No correction |
No compensation is made to the camera settings. |
|
Portrait mode |
Settings are configured for portrait photography. |
|
Landscape mode |
Settings are configured for landscape photography. |
|
Night view mode |
Settings are configured for photography in limited light conditions. |
|
Text mode |
Settings configured for photographing a QR code or other written characters. |
Changing the photo mode overwrites the contrast, gain, sharpness, exposure, and white balance settings as follows, and changes the standard regions for exposure for the outer and inner cameras. When set to All, the start coordinate is (0, 0) with a height of 480 and a width of 640. When set to Center, the start coordinate is (80, 60) with a height of 360 and a width of 480.
Photo Mode |
Contrast |
Gain |
Sharpness |
Exposure |
White Balance |
Outer |
Inner |
---|---|---|---|---|---|---|---|
No correction |
|
Normal |
0 |
0 |
|
All |
Center |
Portrait mode |
|
Normal |
–2 |
0 |
|
Center |
Center |
Landscape mode |
|
Normal |
+1 |
0 |
|
All |
Center |
Night view mode |
|
Maximum |
–1 |
+2 |
|
All |
Center |
Text mode |
|
Normal |
+2 |
+2 |
|
All |
Center |
Flipping
Call the nn::camera::FlipImage
function to specify how to flip the camera's output images. Choose from the following values.
Setting Value |
Flipping to Apply |
---|---|
|
No flipping |
|
Horizontal flipping |
|
Vertical flipping |
|
Rotate image 180° |
Effects
Call the nn::camera::SetEffect
function to specify any special effects to apply to the camera's output images. Choose from the following values.
Setting Value |
Effect |
---|---|
|
No special effects |
|
Monochrome |
|
Sepia tone (ocher) |
|
Negative |
|
Film-tone negative Same as |
|
Sepia tone (red ocher) |
Applying an effect and then changing other settings may change the effect. To give an image a softer feel, apply a sepia effect and then reduce the image's sharpness; to give an image a warmer, redder look, apply the film-tone negative effect and then raise the image's color temperature.
Contrast
Call the nn::camera::SetContrast
function to set the camera's contrast (gamma curve). Choose from the following values.
Setting Value |
Contrast |
---|---|
|
Contrast pattern number |
|
Sets the contrast ratio to higher than the default value (pattern number 7). |
|
Default setting (pattern number 6). |
|
Sets the contrast ratio to lower than the default value (pattern number 5). |
Lens Correction
Lens correction is a means of adjusting differences in brightness that may occur between the center and edges of an image by raising the brightness of the edges to more closely match the center. Call the nn::camera::SetLensCorrection
function to set the camera's lens correction. Choose from the following values.
Setting Value |
Lens Correction |
---|---|
|
Disables lens correction. |
|
Enables lens correction at a value of 70. |
|
Enables lens correction at a value of 90. |
|
Sets edges to be darker than the default setting ( |
|
Default setting ( |
|
Sets edges to be brighter than the default setting ( |
Context
Use contexts to switch multiple photography environment settings at the same time. Use the nn::camera::SwitchContext
function to switch contexts. Each camera has two contexts, A and B, for a total of six sets of settings. These contexts can be switched independently, so the outer camera can be in context A, while the inner camera is in context B.
Each context can specify three settings: resolution, flipping, and special effects.
Noise Filter
When the screen changes from bright to dark or from dark to bright, the camera module applies a noise filter to remove noise from the automatically captured image. The images captured from one of the outer cameras may appear fuzzy when in stereo mode with this noise filter feature enabled. To prevent this, call the nn::camera::SetNoiseFilter
function and pass false
for the on
parameter to disable the noise filter. Pass true
for the on
parameter to enable the noise filter. All cameras have the noise filter enabled by default.
Configuring Photography Environment Settings in Batch
The API provides a feature to simultaneously configure all of the photography environment settings.
Call the nn::camera::SetPackageParameterWithoutContext
function to configure all photography environment settings with no context value specified (other than resolution, flipping, and effects).
Call the nn::camera::SetPackageParameterWithContext
function to configure all photography environment settings with context values specified (resolution, flipping, and effects).
6.4.3. Capture Settings
The images captured by the camera are written line by line in FIFO order, with the YUV-format data sent to a buffer prepared by the application. The application then sends the data to the YUVtoRGB circuit and the resulting RGB-format data to a buffer. The following diagram shows this set of operations.
The following sections describe the settings required to take YUV-format image data through to the application buffer.
About Ports
As with the stereo mode, it is possible to use two cameras at once, making it necessary to specify which camera to use for acquiring captured images. Of the three cameras, the inner camera and the right outer camera are connected to one port, and the left outer camera is connected to its own port.
Most functions that configure image capture require that a port be specified, using the following enumerators.
Enumerator |
Description |
---|---|
|
No port is specified. |
|
Specifies the port to which the inner and right outer cameras are connected. |
|
Specifies the port to which the left outer camera is connected. |
|
Specifies both ports. |
Trimming
Trimming crops a captured image to the size required by the application. Trim an image if the camera's resolution and the required size of the captured image differ.
Call the nn::camera::SetTrimming
function to enable or disable trimming. Call the nn::camera::SetTrimmingParams
or nn::camera::SetTrimmingParamsCenter
functions to specify the range to trim. Call the nn::camera::IsTrimming
function to check whether trimming is currently enabled, and call the nn::camera::GetTrimmingParams
functions to get the trimming range.
Set trimming prior to capturing images.
When setting the trimming position and range using the nn::camera::SetTrimmingParams
function, specify the position to start trimming as (x1, y1) and the position to stop trimming as (x2, y2). The (x1, y1) coordinate is included in the trimming operation, but the (x2, y2) coordinate is excluded. The following limitations apply to these settings.
- The x1 and y1 coordinate values for the trimming start position must be even values.
- x1 must be less than x2, and y1 must be less than y2.
- The post-trimming width (x2 – x1) and height (y2 – y1) must also be even values.
- The post-trimming image width times the height must be a multiple of 128.
When setting the trimming position and range using the nn::camera::SetTrimmingParamsCenter
function, specify the trimming size width and height (trimWidth
and trimHeight
) and the camera resolution width and height (camWidth
and camHeight
) for trimming based on the center of the captured image. The following code gives an example of how the (x1, y1) and (x2, y2) coordinates are calculated. Be sure that the calculated positions meet the restrictions described for the SetTrimmingParams
function.
x1 = (camWidth - trimWidth) / 2; y1 = (camHeight - trimHeight) / 2; x2 = x1 + trimWidth; y2 = y1 + trimHeight;
The captured image size sent to the buffer is the trimmed size. The cost of a transfer is still the same for different camera resolutions, provided you trim the images to the same size. However, trimming high-resolution images may cause the image to appear compressed due to the narrower field of view.
Number of Transfer Bytes (Lines)
The images captured by the camera are stored by the hardware FIFO line by line, and then written with multiple transfers to the buffer prepared by the application. The FIFO capacity is fixed at 10 KB. The number of bytes sent in a single transfer must meet the following conditions.
- The number of bytes sent in a single transfer must be a multiple of 256.
- The number of bytes sent in a single transfer must be no more than 10 KB (10,240 bytes).
- The total number of transfer bytes must be a multiple of the number of bytes sent in a single DMA transfer.
The total number of transfer bytes is the width times the height of the trimmed image, multiplied by 2, which is the number of bytes per pixel.
Call the nn::camera::SetTransferLines
function to set the number of bytes in a single transfer in terms of the number of lines. You can simply set the single-transfer line count to the value returned by the nn::camera::GetMaxLines
function, but note that execution halts in this function if the number of lines does not match the conditions described above. If you cannot set the number of lines, such as when setting an optional resolution using the nn::camera::SetDetailSize
function, instead call the nn::camera::SetTransferBytes
function to set the number of bytes in a single transfer. You can simply set the single-transfer byte count to the value returned by the nn::camera::GetMaxBytes
function, but note that execution halts in this function if the number of bytes (width × height × 2) is not a multiple of 256.
Set the number of transfer bytes (lines) prior to capturing images. To prevent buffer errors, the GetMaxLines
and GetMaxBytes
functions calculate FIFO capacity as 5 KB.
Call the nn::camera::GetTransferBytes
function to get the current balance setting.
Receive Buffer
Call the nn::camera::GetFrameBytes function to get the size of the buffer required for receiving one frame of captured video. The starting address of the buffer is 4-byte aligned. An alignment below 64 bytes may reduce the transfer rate. Only a buffer allocated in device memory may be specified.
6.4.4. Starting Capture
Before transferring or capturing an image, call the nn::camera::Activate
function to activate the camera to use for capture. You cannot activate the inner camera and the outer camera (R) at the same time, because the two cameras are connected to the same port. The outer camera (L) is connected to a different port. You can activate it at the same time as either the inner camera or the outer camera (R).
After making sure that the camera is active, call nn::camera::SetReceiving
to begin transfer, and then call nn::camera::StartCapture
to begin image capture.
void nn::camera::SetReceiving(nn::os::Event* pEvent, void* pDst, nn::camera::Port port, size_t imageSize, s16 transferUnit);
Specify the event for receiving notification that transfer has completed in the pEvent
parameter. Specify the starting address of the receive buffer in pDst
. Specify the port in port
. Specify the byte size of a single frame of captured video (the receive buffer size) in imageSize
. In transferUnit
, specify the byte size of the data in a single transfer as returned by nn::camera::GetTransferBytes
.
You can call nn::camera::IsFinishedReceiving
to get whether the single frame of captured video has finished transferring.
To check whether an error has occurred during capture, such as during FIFO writing, check whether the error event returned by nn::camera::GetBufferErrorInterruptEvent
is in the signaled state. If a buffer error has occurred, this error event is in the signaled state and is an automatically resetting event of the nn::os::Event
class. This error event also changes to a signal state when a camera error causes the camera to restart. Recover from an error by restarting in the order of transfer, and then capture.
You can call nn::camera::IsBusy
to find out whether the camera is currently capturing an image. You can also call nn::camera::GetVsyncInterruptEvent
to get the event that enters the signaled state when the camera's VSYNC interrupt occurs. Use this function in a process that synchronizes with camera VSYNC, or in a process that changes the camera environment when a frame is not being transferred.
Call nn::camera::SynchronizeVsyncTiming
to synchronize the camera VSYNC interrupt timing, such as when you are using the stereo camera to display in stereoscopic view. This function attempts to synchronize the VSYNC interrupt timing of the two cameras. This process does not maintain the synchronization. The function must be called again when synchronization is lost. The four frames after calling this function may be very dark, because it places the camera into standby before restarting it. The two cameras have a small gap in the timing of the image capture being sent, even if the cameras have the same configuration. VSYNC interrupt timing has a large gap after calling nn::camera::Activate
to start the camera, or when changing the camera configuration even when capturing video with the same configuration and fixed frame rate.
The capture image (YUV4:2:2 format) is output in the following alignment.
+ 0 Byte |
+ 1 Byte |
+ 2 Byte |
+ 3 Byte |
---|---|---|---|
Y (n) |
U (n) |
Y (n + 1) |
V (n) |
6.4.5. YUVtoRGB Circuit Settings
Use the YUVtoRGB circuit to convert YUV-format data to RGB format at the hardware level. This circuit also supports output to the native GPU block format. However, the CTR system is only equipped with one YUVtoRGB circuit. When converting captured images from multiple cameras, such as when in stereo mode, use a mutex or other mechanism to make sure that multiple conversion requests are not issued simultaneously.
This section describes settings related to data conversion using the YUVtoRGB circuit.
Input Format
Call the nn::y2r::SetInputFormat
function to set the input YUV data format. Call the nn::camera::GetInputFormat
function to get the current setting. Choose from the following values.
Setting Value |
Format |
---|---|
|
Input the individual Y, U, and V values for YUV4:2:2 as 8 bits each. |
|
Input the individual Y and U values for YUV4:2:0 as 8 bits each. |
|
Input the individual Y, U, and V values for YUV4:2:2 as 16 bits each (padding required). |
|
Input the individual Y and U values for YUV4:2:0 as 16 bits each (padding required). |
|
Input the Y, U, and V values for YUV4:2:2 all together in 32 bits. |
The data formats for each value are shown below.
YUV4:2:2/YUV4:2:0 individual input (8 bit)
Component |
+ 0 Byte |
+ 1 Byte |
+ 2 Byte |
+ 3 Byte |
---|---|---|---|---|
Y |
Y (n) |
Y (n + 1) |
Y (n + 2) |
Y (n + 3) |
U |
U (n) |
U (n + 1) |
U (n + 2) |
U (n + 3) |
V |
V (n) |
V (n + 1) |
V (n + 2) |
V (n + 3) |
YUV4:2:2/YUV4:2:0 individual input (16 bit padding)
Component |
+ 0 Byte |
+ 1 Byte |
+ 2 Byte |
+ 3 Byte |
---|---|---|---|---|
Y |
Y (n) |
padding |
Y (n + 1) |
padding |
U |
U (n) |
padding |
U (n + 1) |
padding |
V |
V (n) |
padding |
V (n + 1) |
padding |
YUV4:2:2 batch input
Component |
+ 0 Byte |
+ 1 Byte |
+ 2 Byte |
+ 3 Byte |
---|---|---|---|---|
YUV |
Y (n) |
U (n) |
Y (n + 1) |
V (n) |
You can only get captured images from the camera in YUV batch format. In most cases, specify INPUT_YUV422_BATCH
when converting captured images.
Line Width and Line Count
Call the nn::y2r::SetInputLineWidth
function to specify the line width of the data to convert (the input data). Call the nn::y2r::GetInputLineWidth
function to get the current setting. The line width must be set to a multiple of 8, up to a maximum value of 1024.
Call the nn::y2r::SetInputLines
function to set the number of input lines. Call the nn::y2r::GetInputLines
function to get the current setting.
Output Format
The data converted by the YUVtoRGB circuit is stored in the output buffer. Call the nn::y2r::SetOutputFormat
function to set the output data format. Call the nn::camera::GetOutputFormat
function to get the current setting. Choose from the following output formats.
Setting Value |
Format |
---|---|
|
32-bit RGB (RGBA8888) |
|
24-bit RGB (RGB888) |
|
16-bit RGB (RGBA8888) |
|
16-bit RGB (RGB565) |
The data formats for each value are shown below.
32-bit RGB (OUTPUT_RGB_32
)
24-bit RGB (OUTPUT_RGB_24
)
16-bit RGB (OUTPUT_RGB_16_555
)
16-bit RGB (OUTPUT_RGB_16_565
)
Formats that output the alpha component use the alpha value that was set by the nn::y2r::SetAlpha
function. The OUTPUT_RGB_32
format uses bits 0 to 7 of the set alpha value, while the OUTPUT_RGB_16_555
format only uses the 7th bit. Call the nn::camera::GetAlpha
function to get the current alpha value.
Block Alignment
Call the nn::y2r::SetBlockAlignment
function to set the block alignment for the data stored in the output buffer. Call the nn::camera::GetBlockAlignment
function to get the current setting. Choose from the following block alignments.
Setting Value |
Block Alignment |
---|---|
|
Horizontal line format. The usual linear format. The 24-bit and 32-bit RGB output formats cannot be used directly as OpenGL standard-format textures due to byte-order issues. |
|
8x8 block format. This is the native GPU block format. Data in this format can be used as native-format textures. |
When using BLOCK_8_BY_8
block alignment, the input image height (vertical line count) must be a multiple of 8.
Output Buffer
You can call the nn::y2r::GetOutputImageSize
function to get the size of the buffer required for receiving one frame of output data. The starting address of the buffer is 4-byte aligned. An alignment below 64 bytes may reduce the transfer rate. Only a buffer allocated in device memory may be specified.
Conversion Coefficient
Choose the coefficient for YUV to RGB conversion from the following standard conversion coefficients. Call the nn::y2r::SetStandardCoefficient
function to set the conversion coefficient.
Select the type of conversion coefficient for converting images output by the cameras from those returned by the nn::camera::GetSuitableY2rStandardCoefficient
function. When making a selection, consider the possibility of future changes in the camera module.
You can choose from the following four types of standard conversion coefficients.
Setting Value |
Conversion Coefficient Type (Value Range) |
---|---|
|
ITU-R BT.601 (0 ≤ Y, U, V ≤ 255) |
|
ITU-R BT.709 (0 ≤ Y, U, V ≤ 255) |
|
ITU-R BT.601 (16 ≤ Y ≤ 235, 16 ≤ U, V ≤ 240) |
|
ITU-R BT.709 (16 ≤ Y ≤ 235, 16 ≤ U, V ≤ 240) |
Rotation
You can rotate an image when converting the format. To specify how many degrees to rotate an image, call the nn::y2r::SetRotation
function; to get the current rotation degree setting, call the nn::y2r::GetRotation
function. You can choose from the following four rotation settings, including no rotation.
Setting Value |
Rotation Angle |
---|---|
|
No rotation. |
|
Clockwise 90° |
|
180° |
|
Clockwise 270° (counterclockwise 90°) |
When you rotate an image, the post-conversion data no longer has valid image data ordering. Consequently, your application must correct the data ordering after receiving each frame.
This may be changed in the future so that valid data ordering is output by transfer.
Batch Configuring
Call the nn::y2r::SetPackageParameter
function to configure all of the YUVtoRGB circuit settings at the same time. You can get all the settings at the same time using the nn::y2r::GetPackageParameter
function.
6.4.6. Starting Format Conversion
Data conversion is carried out in parallel with sending and receiving, so preparations for sending and receiving data must be completed before conversion is begun.
Use the following functions to prepare to send input data. nn::y2r::SetSendingYuv
for batched YUV data. nn::y2r::SetSendingY
for just the Y data. nn::y2r::SetSendingU
for just the U data. nn::y2r::SetSendingV
for just the V data. These functions take the input data buffer's size, the total transfer data size, and the size of one line of input data, specified in bytes, as arguments. Only a buffer allocated in device memory may be specified. Note that the total transfer data size must be a multiple of the size of one line. The offset value added when one line of input data is transferred (transferStride
) may be specified for any of these functions.
Call nn::y2r::SetReceiving
to prepare to receive output data. In the arguments, specify the buffer storing the output data, the total size of the data to receive, and the transfer size of a single chunk of received data in bytes. You cannot specify a buffer in VRAM. Only a buffer allocated in device memory may be specified. To improve performance, we recommend specifying the transfer size as the size of eight lines of output data. Call the nn::camera::GetOutputImageSize
function to get the current setting. The size value specified for eight lines must be the size of a single pixel in bytes returned by the nn::y2r::GetOutputFormatBytes
function multiplied by the width of a single line multiplied by 8. You can also specify an offset value added to each transfer (transferStride
). To add an offset to each line, specify the size of one line of output data for the size of a single transfer.
A good time to prepare to send and receive data is immediately after the event specified by the nn::camera::IsFinishedReceiving
or nn::camera::a SetReceiving function
is in the signaled state, which indicates that the captured image has been fully received and you have confirmed the input data. Before preparing to send or receive data, call the nn::y2r::IsBusyConversion
function to check whether the system is busy converting formats.
Call the nn::y2r::StartConversion
function to begin format conversion after data send/receive preparations are complete. Data is sent and received at the same time as format conversion begins. Call the nn::y2r::StopConversion
function if you must forcibly halt format conversion, such as when an error occurs during capture operations.
Call the nn::y2r::IsBusyConversion
function to check whether the system is busy converting formats. To check whether the system has completed sending input data, call the nn::y2r::IsFinishedSendingYuv
function for batched YUV data, the nn::y2r::IsFinishedSendingY
function for just the Y data, the nn::y2r::IsFinishedSendingU
function for just the U data, and the nn::y2r::IsFinishedSendingV
function for just the V data. To check whether the system has finished receiving output data, call the nn::y2r::IsFinishedReceiving
function.
You can use a combination of these functions to check whether format conversion and data transmission has completed for a frame. You can also check whether data transfer is complete by using the event class (automatically resetting event) returned by the nn::y2r::GetTransferEndEvent
function. This event class receives an interrupt notification when transfer has completed. The nn::y2r::SetTransferEndInterrupt
function must be called to enable interrupt notifications before getting the event class. (The default is to disable them.) Call the nn::y2r::GetTransferEndInterrupt
function to check whether notifications are currently enabled.
A hardware bug prevents the transfer completion event obtained by the nn::y2r::GetTransferEndEvent
function from being signaled, and prevents the nn::y2r::IsBusyConversion
function from returning true
, if the following (rare) sequence occurs: (1) while using the cameras and YUVtoRGB circuit simultaneously, (2) a buffer error occurs during camera data transfer, and (3) during recovery from that error the processing of the nn::camera::SetReceiving
function overlaps the YUVtoRGB circuit data transfer processing at a particular time. For this reason, always make sure to specify a timeout when waiting for the signal status of a transfer completion event.
Specify a timeout longer than the time required for conversion. The time required for conversion depends on the size of the input image and the output format. Output of a VGA image in 16-bit RGB format takes about 13 milliseconds. Output of a VGA image in 24-bit RGB format takes about 16 milliseconds.
If this bug occurs, forcibly terminate conversion by calling the nn::y2r::StopConversion
function. This makes the YUVtoRGB circuit usable again.
The probability of occurrence of this bug is proportional to the frequency of camera buffer errors, so use the camera under conditions where buffer errors are unlikely to arise. When buffer errors occur frequently, respond by lowering the camera frame rate or with another method. Also, set a higher priority for the thread that calls the nn::camera::SetReceiving
function.
6.4.7. Playing the Shutter Sound
The shutter sound is forcibly played even when the speaker volume is set to 0 and the camera LED is temporarily turned off.
SNAKE and FTR do not have a camera indicator LED.
nn::Result nn::camera::PlayShutterSound(nn::camera::ShutterSoundType type);
Specify one of the following shutter sound types for the type
parameter.
Setting Value |
Shutter Sound Type |
---|---|
|
Normal shutter sound. |
|
Sound played when starting video capture. |
|
Sound played when ending video capture. |
6.4.8. Ending Capture
To stop image capture, do the following. Stopping image capture without following the steps below can lead to audio noise when displaying the HOME Menu.
- Call the
nn::y2r::StopConversion
function to stop format conversion. - Call the
nn::camera::StopCapture
function to stop capture. - Call
nn::camera::Activate(SELECT_NONE)
to put all cameras into standby mode. - Call the
nn::camera::Finalize
andnn::y2r::Finalize
functions to close the camera and Y2R libraries.
6.4.9. Support for Sleep Mode
nn::camera::ResultIsSleeping
(or nn::y2r::ResultIsSleeping
) is returned to indicate the system is closed when a Camera
(or Y2R) library function is called while the system is closed. This includes during initialization. Note that this return value is returned as long as the system is closed, even if it is not in Sleep Mode.
The camera is designed so that operations stop if the system is closed, regardless of Sleep Mode. If the system is closed while capturing an image, capture resumes when the system is opened. Note, however, that when capture resumes there is a period of time immediately after the camera is activated by the Activate
function where the image is unstable. This is due to processing equivalent to the nn::camera::Activate(SELECT_NONE)
function being performed inside the library when the system is closed. There is also a possibility that the system will enter a state where the IsBusy
function always returns true
depending on the exact timing the system was closed. This status will be canceled when the system is opened, but do not perform polling using the IsBusy
function if a process for entering Sleep Mode when the system is closed has been implemented.
If the system enters sleep in the midst of RGB conversion by the Y2R library, conversion is forcibly terminated. Conversion does not resume after recovering from sleep. If supporting sleep, do not enter sleep during conversion (when the IsBusyConversion
function returns true
or while waiting for an event obtained by the GetTransferEndEvent
function). Implement code so that the system only enters sleep after checking that conversion is complete. Particularly note that events will not enter signal status after recovering from sleep if sleep is entered while waiting for an event obtained using the GetTransferEndEvent
function.
6.4.10. Conflicts With Camera Mode
You cannot press the L Button and the R Button simultaneously to enter camera mode from the HOME Menu while an application is using the cameras. If you attempt to do so, a dialog box is displayed with a message stating that the cameras are in use.
The cameras are considered to be in use if either the CAMERA or the Y2R library has been initialized. If an application that uses the cameras initializes either of these libraries when it is started, the cameras are considered to be in use when the HOME Menu is displayed even if the application is not actually using them. Even if the Y2R library alone is used to convert YUV images into RGB images—during movie playback, for example—the cameras are considered to be in use and the system does not enter camera mode.
Initialize the CAMERA and Y2R libraries just before you use them and shut them down afterwards whenever you can. Do not leave the CAMERA and Y2R libraries initialized while they are not in use.