Embedded Linux Kernel Error Tracking Technology

RF cable can be customized for other specifications

Five authorized agents - original - spot technical support

With the wide application of embedded Linux systems, higher requirements are placed on the reliability of the system, especially in important areas such as life and property, which require the system to achieve a safety integrity level of 3 or above [1], failure rate (per The probability of a dangerous failure in an hour is 10-7 or less, which is equivalent to a system with a mean time between failure (MTBF) of at least 1141 years. Therefore, improving system reliability has become an arduous task. A survey of the application of a company's 14 878 controller systems in the industrial field shows that from the beginning of 2004 to the end of September 2007, with the continuous improvement of hardware and software, the failure rate according to the error report has been reduced to five-fifths in 2004. One or less, but the time to find the error has increased by more than three times.

The time required to solve the problem is rising, although there are software problems, but the lack of necessary means to help solve the problem is the main reason. Through the statistical tracking of faults, it is found that software errors that are difficult to solve and software errors that are long-term from discovery to solution are concentrated in the core part of the operating system, and a large proportion of them are concentrated in the driver part [2]. Therefore, error tracking technology is seen as an important measure to improve the system's security integrity level [1]. Most modern operating systems provide development system kernel "crash dump" mechanism, that is, when the software system is down Save the contents of the memory to disk [3], or send it to the failed server [3] via the network, or directly start the kernel debugger [4], etc., for post-mortem analysis improvement.

The crash dump mechanism based on the Linux operating system kernel has the following several types in recent years:

(1) LKCD (Linux Kernel Crash Dump) mechanism [3];

(2) KDUMP (Linux Kernel Dump) mechanism [4];

(3) KDB mechanism [5];

(4) KGDB mechanism [6].

Combining the above several mechanisms, we can find that there are three common points between these four mechanisms:

(1) Applicable to applications with rich computing resources and sufficient storage space;

(2) There is no strict requirement for recovery time after a system crash;

(3) Mainly for more general hardware platforms, such as the X86 platform.

In an embedded application, if you want to use one of the above listed mechanisms directly, you will not be able to solve the following three difficulties:

(1) Insufficient storage space

Embedded systems generally use Flash as the memory, and Flash has limited capacity and may be much smaller than the memory capacity in embedded systems. Therefore, saving all memory contents to Flash is not feasible.

(2) Recording time requirements are as short as possible

Embedded systems generally have the requirement of reset response time as short as possible. Some embedded operating system reset restart time does not exceed 2s, and the above-mentioned kernel crash dump mechanism that can be used for Linux system can not take up to 30s. Writing Flash is also time consuming. Experiments show that writing 2MB of data to Flash takes up to 400ms.

(3) Requires support for a specific hardware platform

The hardware of the embedded system is various. The four mechanisms mentioned above provide better support for the X86 platform, while the hardware support for other systems is not mature.

Due to these difficulties, it is very difficult to port one of the above four kernel crash dump mechanisms to a specific embedded application platform. Therefore, in view of the three characteristics of the above embedded system, this paper introduces a platform-based embedded Linux kernel crash information recording mechanism LCRT (Linux Crash Record and Trace), to locate software faults and solve software faults in embedded Linux systems. Provide assistance.

1 analysis of Linux kernel crash

Analysis of the Linux kernel for the processing of various "traps" during the run, you can know that the Linux kernel can monitor the errors caused by the application, in the case of application zero-deletion, memory access violations, buffer overflows, etc., the Linux kernel Exception handling routines can handle these exceptions caused by the application. When an application generates an unrecoverable error, the Linux kernel can simply terminate the application that generated the error, and other applications will still function.

If there is a bug in the Linux kernel itself or the newly developed Linux kernel module, errors such as "division by zero", "memory access out of bounds", "buffer overflow", etc., will also be handled by the Linux kernel exception handling routine. The Linux kernel judges in the exception handler that if it finds a "serious unrecoverable" kernel exception, it will cause "kernel panic", which is the Linux kernel crash. Figure 1 shows the processing flow of the Linux kernel for exceptions.