Ingenic ISP Tuning Parameters, Explained

Every Ingenic camera ships with a handful of hidden tweaks under the hood, settings with names like isp_memopt and isp_ch0_pre_dequeue_time that sit in a config file rather than a menu. The defaults are fine, until they are not: a camera that runs out of memory and drops the feed, a stream that stutters under load, a log that drowns everything useful in routine chatter. When one of those happens, these are the knobs you reach for. They all live in the same place, so it is worth a quick look at that first.

Where these settings live

All of these settings arrive as name=value pairs on a single line, written into a file on the camera at build time. A T31 loads them like this:

tx_isp_t31 isp_clk=200000000 isp_day_night_switch_drop_frame_num=0 \
  isp_ch0_pre_dequeue_time=24 isp_ch0_pre_dequeue_interrupt_process=0 \
  isp_memopt=1 print_level=1

The kernel driver tx-isp reads the line and accepts each pair. In the driver source, a pair is declared like this:

module_param(print_level, int, S_IRUGO);
MODULE_PARM_DESC(print_level, "isp print level");

One thing to know before going any further. The actual image-processing code inside the ISP is a closed binary supplied by Ingenic; only the thin outer layer is open source. The open code tells us a knob exists, and it tells us its name, but often not what the knob does inside. From here on, the excerpts show what the open code says; the plain-English explanation is the part that had to be reasoned out.

isp_memopt

This setting controls how much memory the ISP is allowed to use. Think of the ISP as a workshop with several workstations in a row. The picture moves along the line, and each workstation does one small job. Every workstation needs storage space to hold the picture while it works on it.

On a cheap camera chip, there is very little RAM. The isp_memopt setting tells the ISP which workstations to switch off, so the ones that remain have enough space to do their job. Switching a workstation off saves memory, but it also means that particular bit of picture polish does not get done.

The code names three workstations it can switch off:

/* 0:nomal 1:bypass Lynne+BGM 2:bypass Lynne+BGM+Ass 3-f:nomal */
static int isp_memopt = 1;

Lynne, BGM, and Ass are the names Ingenic gave to three of the workstations; nobody explains what the names stand for, so we treat them as labels. What matters is the trade-off:

The setting is actually two numbers packed into one: the low part controls the main path, the high part controls a second path used by cameras with two sensors. A normal single-sensor camera just uses 0, 1, or 2.

The build system sets 1 automatically for any chip with 64 MB of RAM or less. That is the sensible trade-off: lose a little polish, gain a lot of room.

isp_ch0_pre_dequeue_time

This setting controls how early the ISP hands over each picture. The ISP produces pictures one after another, like frames off a film reel. When a frame is finished, the ISP must hand it to the video encoder, which compresses it into a stream. This hand-over takes a little time.

The ISP only knows a frame is completely finished when it has drawn the last row of the picture. If it waits until that exact moment before starting the hand-over, the encoder sits idle for the whole duration of the hand-over, and the pipeline hiccups.

This setting fixes that. It says: start the hand-over this many milliseconds before the frame is officially declared finished. The picture data is already nearly all there, so starting early is safe, and it means the encoder can get going without waiting.

MODULE_PARM_DESC(isp_ch0_pre_dequeue_time, "isp pre dequeue time, unit ms");

A value of 24 means the hand-over starts 24 ms early. A bigger number hides more of the pause, but if it is too big, the hand-over can start before the last rows exist, which shows up as a torn or corrupted strip at the bottom of the picture. A value of 0 turns the feature off and behaves the old way.

Channel 0 is the full-resolution output, the main stream. Channel 1 is the downscaled sub stream. Both are always present.

isp_ch0_pre_dequeue_interrupt_process

This setting controls the extra work allowed during the early hand-over. While the ISP is doing its normal work, it occasionally gets interrupted by small urgent jobs. This setting controls whether those urgent jobs are allowed to run during the early hand-over window described above.

MODULE_PARM_DESC(isp_ch0_pre_dequeue_interrupt_process, "isp pre dequeue interrupt process");

A value of 0 means no - during the early hand-over, the ISP focuses only on handing the frame over, and nothing interrupts it. This is the simplest and safest choice for a camera with one sensor, and it is what Thingino ships by default.

A nonzero value allows some urgent jobs to run alongside the hand-over. That is useful only in special cases, such as a sensor or multi-camera setup that needs its interrupts handled earlier to keep everything in sync.

print_level

This setting controls how much the ISP talks. The ISP can print messages to the kernel log about what it is doing. It controls how chatty it is. The levels are spelled out in the open source (tx-isp-debug.h):

#define ISP_INFO_LEVEL    0x0
#define ISP_WARNING_LEVEL 0x1
#define ISP_ERROR_LEVEL   0x2

Messages have a level. A message is printed only if its level is at or above the print_level setting.

Thingino defaults to 1 everywhere except the old T10/T20/T21 chips. That gives warnings and errors in the log without drowning in routine info.

One quirk: when an error gets printed, the driver also prints a stack trace, a dump of what the code was doing at that moment. So level 2 produces errors plus a stack trace for each one.

isp_ch0_pre_dequeue_valid_lines

This setting defines what counts as “finished”. Recall that the early hand-over starts before the frame is officially done. It needs a rule for “done”. This setting supplies the rule: it tells the ISP how many picture rows must be valid before the frame counts as complete.

MODULE_PARM_DESC(isp_ch0_pre_dequeue_valid_lines, "isp pre dequeue valid lines");

A picture is drawn row by row, from the top down. The ISP counts the rows as they arrive. When it has seen this many valid rows, it declares the frame finished and hands it over. The value is a number of rows, and it pairs with the pre-dequeue time from earlier: the time says how early to reach for the frame, and this says how much of the frame must exist before the reach is legal.

isp_ch1_dequeue_delay_time

Channel 1 is the downscaled sub stream that every camera produces alongside the full-resolution main stream. This setting is the channel 1 version of the early hand-over timing.

MODULE_PARM_DESC(isp_ch1_dequeue_delay_time, "isp pre dequeue time, unit ms");

It is measured in milliseconds and behaves the same way as isp_ch0_pre_dequeue_time, but for the sub stream.

isp_day_night_switch_drop_frame_num

This setting controls the frames tossed during the day/night changeover. Most cameras switch between a color picture for daytime and a black-and-white picture (with an infrared lamp) for nighttime. At the exact moment of the switch, the image sensor is still settling: colors are wrong, the exposure is wild, the first frames are garbage.

This setting tells the ISP how many frames to throw away at that moment, before any reach the video stream.

MODULE_PARM_DESC(isp_day_night_switch_drop_frame_num, "isp day night switch drop frame number");

A value of 0 drops nothing, so the first ugly frames appear in the stream. A small number skips just the worst of them, keeping the switch brief. A large number hides the mess entirely but adds a short freeze to the picture.

direct_mode

This setting controls how the ISP hands frames to the encoder. The ISP and the video encoder are two separate pieces of hardware. Normally the ISP writes each finished frame into a shared memory buffer, and the encoder reads it back out again - a stop at a relay station between two trains. This setting (present on the T23 and T41) changes that hand-off.

MODULE_PARM_DESC(direct_mode, "isp direct mode");

The direct modes are faster and use less memory because they skip the buffer copy, but they only work when the two chips are set up to talk directly to each other. The full answer involves IVDC, below.

IVDC

IVDC is the short cut between the ISP and the encoder. The name is short for ISP-VPU Direct Connect. “VPU” is the video processing unit, the hardware that does the compressing. IVDC is a direct cable between the ISP and the VPU: instead of the ISP writing a frame to memory and the VPU reading it back, the frame flows straight from one to the other.

That saves a full frame’s worth of memory and a copy step, which matters a lot on a camera with two sensors, where everything is done twice. Two settings control how much of the picture IVDC holds at once.

ivdc_mem_line

This setting controls how many rows IVDC parks in memory. Even a direct connection needs a little staging room, because the ISP and the VPU do not move perfectly in lockstep. This setting decides how many rows of the picture IVDC keeps parked in memory (DDR) while it transfers them.

MODULE_PARM_DESC(ivdc_mem_line, "ivdc mem line");

A row here is one full width of the picture, rounded up to a multiple of 256 pixels. The value 0 (the default) means “store a whole frame”, which is the safe, memory-hungry choice. A value greater than zero caps the storage at that many rows, trading less memory for tighter timing. This setting exists on the T23 and T41, and only matters on a multi-sensor camera.

ivdc_threshold_line

This setting controls how much IVDC buffers before it flushes. Where ivdc_mem_line sets the maximum rows stored, ivdc_threshold_line sets the amount that must build up before IVDC moves it along. It is a cache size, measured in rows, rounded to a multiple of 256 pixels.

MODULE_PARM_DESC(ivdc_threshold_line, "ivdc threshold line");

The value 0 (the default) sets the threshold to half a frame. A value greater than zero sets it to that many rows. Bigger thresholds move data in larger chunks, which is more efficient but needs more memory; smaller thresholds move it sooner. Like ivdc_mem_line, it is a multi-sensor setting on the T23 and T41.

mipi_switch_gpio

This setting names the pin that selects the active sensor. A two-sensor camera does not run both sensors at full speed all the time. It switches between them, and the switch is a physical wire controlled by a GPIO pin on the chip.

mipi_switch_gpio=7

It exists only on the T23 and only when the camera has more than one sensor. The value is the GPIO number, usually 7.

isp_clk, isp_clka, isp_clks

These are three clock speeds for three different parts of the ISP. The ISP is not one circuit running at one speed. It has three separate clocks:

Each clock can be sourced from one of the chip’s clock generators (SCLKA, MPLL, VPLL, and so on) and set to a speed. These are calculated and wired up by the build system, so they are not normally set by hand. They show up in the load line because the driver needs to be told what to run at, and because tuning a camera for a specific sensor sometimes means nudging one of them.

The T30 knobs

The T30 predates the modern ISP, and its settings are much blunter: plain numbers for the output size and crop rectangle, and buffer counts for the LDC (lens distortion correction) stage. These are not worth memorizing unless the camera uses a T30; they are listed here for completeness.

Changing the settings

Everything above is one trade-off after another: memory for polish, latency for safety, chatter for silence. Most of the time the build system already picked the right side of each trade for the chip’s RAM size, and the defaults are correct for a single-sensor camera. The settings exist for the cases where they are not.

Settings are controlled through menuconfig options in Config.soc.in, under “ISP Kernel Module Configuration”. Two things must be set for each: the switch that enables it, and the value. For example, on a T31:

BR2_ISP_CH0_PRE_DEQUEUE_TIME=y
BR2_ISP_CH0_PRE_DEQUEUE_TIME_VALUE=24
BR2_ISP_CH0_PRE_DEQUEUE_INTERRUPT_PROCESS=y
BR2_ISP_CH0_PRE_DEQUEUE_INTERRUPT_PROCESS_VALUE=0
BR2_ISP_PRINT_LEVEL_1=y

Settings with no dedicated menu option can be passed as raw text through BR2_ISP_PARAMS, which is added to the load line as-is.

To check what is actually loaded, on the camera:

cat /etc/modules.d/20-isp
cat /sys/module/tx_isp_t31/parameters/isp_memopt

The module is named tx_isp_<soc>, so change the t31 to match the chip.

To see what the ISP is actually doing at runtime, read the companion piece Reading /proc/jz/isp/isp-m0: your camera’s health dashboard, which decodes the live status dashboard these parameters feed into.