Aggregator
Java动态类加载,当FastJson遇上内网
Java动态类加载,当FastJson遇上内网
Java动态类加载,当FastJson遇上内网
Java动态类加载,当FastJson遇上内网
Java动态类加载,当FastJson遇上内网
Java动态类加载,当FastJson遇到内网
Is the Cloud Safe? Part 2: Breach Highlights for the Past 3 Years
Is the Cloud Safe? Part 2: Breach Highlights for the Past 3 Years
Is the Cloud Safe? Part 2: Breach Highlights for the Past 3 Years
2019 总结
网络空间安全时代的红蓝对抗建设
青藤云Webshell查杀绕过
上周看到一些朋友在发青藤云的聊天记录,绕过一个发200块红包。周末的时候就简单做了个测试。
首先使用绕过D盾的webshell,发现被标记为恶意。这就有点意思了。我去看了一下聊天记录的内容,看原理是采用的深度学习的方式。猜测是有监督的方式来实现的(仅为猜测)。我这里绕过的思路采用的是,使用开源程序进行修改,带入一句话后门代码。
记得Phpmyadmin的index.php文件中包含有include ROOT_PATH . $_REQUEST['target'];这样的代码,如果不看上下文,这个妥妥的文件包含漏洞。不过上文有过滤和判断处理,导致无法包含任意文件。我将其修改为包含任意文件。同时保留来一些源码,把影响执行的代码注释掉(因为适应多种环境,适用但文件的情况)。
最终修改的代码如下:
日志分析系列(外传二):Nginx日志统一化
日志分析系列(外传二):Nginx日志统一化
Laravel 的 Facades 实现原理
日志分析系列(外传二):Nginx日志统一化
固件提取系列-UBI文件系统提取以及重打包
Firmware Extraction Series: Firmware Media
Firmware, sometimes referred to as a firmware image (or simply “ROM” in mobile communities), resides in Non-Volatile Memory (NVM) and can be both read and written. In embedded systems, the most common NVM types are ROM (Read-Only Memory) and Flash memory. While strictly speaking, “ROM” includes Mask ROM, PROM, EPROM, and EEPROM, modern “mainstream ROM” usually refers to EEPROM integrated within an MCU. Flash memory typically serves as the primary external storage.
In embedded devices, beyond standard NAND or NOR flash chips, you may also encounter eMMC. For expandable storage, devices might use SD cards, CF cards, USB drives, or HDDs. These storage solutions generally follow a controller + storage architecture: a controller bridges the host and the storage medium. As long as you can interact with the controller, you can read or write to the underlying storage.
Devices like eMMC, SD cards, and HDDs expose standard external interfaces, allowing them to be read using standard card readers or programming sockets. In contrast, raw flash chips are managed directly by the SoC (System on Chip) via specific drivers; they lack a generic external interface. However, because these chips are memory-mapped peripherals, you can interact with them if you can access their address space. This is the principle behind techniques like reading firmware via JTAG, IAP (In-Application Programming), or bootloaders (like U-Boot). Theoretically, even if the device’s main controller is non-functional, the firmware can still be recovered directly from the storage chip.
For the purpose of this series, I define “firmware” as the original files containing the operating system and data stored on these media. In embedded security research, firmware extraction is almost always the first—and most critical—step. Without the firmware, further research often hits a dead end. This series aims to systematically share the knowledge and techniques I’ve accumulated over years of performing firmware extraction.
EEPROM vs. NOR vs. NAND FlashEEPROM typically offers much higher endurance (erase/write cycles) than Flash memory. Combined with small package sizes and low write/erase power consumption, EEPROM is often the preferred choice for storing configuration data, particularly in automotive applications.
For high-performance storage, Flash memory is the standard. There are two main types:
- NOR Flash: Supports XIP (eXecute In Place) and offers fast read speeds, but provides slower write and erase operations. It supports random access, making it ideal for code execution.
- NAND Flash: Accessed in blocks rather than randomly. It offers significantly higher capacity, higher throughput, and lower cost per bit, but generally has lower reliability and requires complex management.
To communicate with these media, you must speak their protocols:
- EEPROM: Typically uses I2C or SPI (Serial Peripheral Interface).
- NOR Flash: Serial NOR usually uses SPI; Parallel NOR uses a parallel bus. Protocol standards include JEDEC SFDP (JESD216) for SPI and JEDEC CFI (JESD68) for parallel NOR.
- NAND Flash: Uses the Raw NAND protocol, with most modern devices adhering to the ONFI (Open NAND Flash Interface) standard.
Note: JEDEC (Joint Electron Device Engineering Council) defines standards that allow software to query a Flash chip’s manufacturer and device IDs to determine its size and capabilities. However, not all chips strictly adhere to these standards.
NOR Flash PackagesNOR flash is available in parallel and serial variants.
- Serial NOR: Commonly packaged as SOP (Small Outline Package) and uses SPI.
- Parallel NOR: Typically uses TSOP (Thin Small Outline Package) like TSOP-56, or BGA (Ball Grid Array) like TFBGA-56 and LFBGA-64.
Because NOR flash supports random access, it functions similarly to SRAM. Below is the pinout for a parallel NOR flash chip. Manually wiring these (using “flying leads”) can be tedious due to the high pin count.
Symbol Pin Name Function A[MAX:0] Address Address bits for read/write operations DQ[7:0] Data I/O Inputs/Outputs for commands and data DQ[14:8] Data I/O Inputs/Outputs for commands and data DQ15/A-1 Data I/O Data or address input (for x8/x16 mode switching) BYTE# Byte/Word Select Selects 8-bit or 16-bit data organization CE# Chip Enable Activates the device RE# Read Enable Data is valid on the falling edge of this pulse OE# Output Enable Drives data onto the bus when LOW; high-impedance when HIGH WE# Write Enable Triggers write operations WP# Write Protect Prevents unintended program/erase operations when LOW RST# Reset Resets the device RY/BY# Read/Busy Output LOW during operations; HIGH when ready. Requires a pull-up resistor (open-drain). Vcc Power Supply Voltage (typically 3.3V or 1.8V) Vss Ground Ground NC No Connection Unconnected pinTSOP-56 Pinout:
NAND FlashNAND flash is a type of non-volatile storage optimized for high density. Embedded devices commonly use SLC (Single Level Cell) NAND, which stores 1 bit per cell.
Flash memory uses a floating-gate transistor structure. Electrons are trapped in an insulated floating gate to store data. A key characteristic of Flash is that it cannot support in-place overwrites. Writing involves capturing electrons, but “erasing” involves releasing them. To erase, a high voltage is applied to pull electrons from the floating gate. Because the source connections are grouped, erasure must happen in large blocks, not individual bytes.
NAND PackagesThe ONFI standard defines several common packages for NAND flash, typically using SMT (Surface Mount Technology).
TSOP-48:
BGA-63:
NAND Flash Pin AssignmentNAND flash uses a multiplexed parallel I/O interface, typically 8-bit (x8). Pins marked with # are active-low and usually require pull-up resistors.
Symbol Pin Name Function I/O x Data I/O Used for command, address, and data input/output. High-impedance when disabled. CLE Command Latch Enable When HIGH, commands are latched on the rising edge of WE#. ALE Address Latch Enable When HIGH, addresses are latched on the rising edge of WE#. CE# Chip Enable Activates the device. When marked high, the device enters standby. RE# Read Enable Data is driven onto the bus on the falling edge of this pulse. WE# Write Enable Latches data/address/commands on the rising edge. WP# Write Protect Hardware write protection. R/B# Read/Busy Indicates device status. LOW = Busy; HIGH = Ready. Open-drain output (requires pull-up). Vcc Power Supply Voltage (3.3V / 1.8V) Vss Ground Ground NC No Connection Unconnected Array OrganizationNAND is organized hierarchically. Below is the organization of an 8-bit NAND chip from ESMT:
- Page: 2048 bytes (Data) + 64 bytes (Spare/OOB)
- Block: 64 Pages
- Device: 1024 Blocks
- Total Capacity: 1056 Mbits (128 MB Data + 4 MB Spare)
For non-expandable storage (soldered chips), reading methods fall into three categories:
- Chip-off (Offline): Desolder the chip and read it using a dedicated programmer and socket.
- Pros: Direct access, works if the device is dead.
- Cons: Higher cost (hardware), potential for damage, requires handling ECC/bad blocks manually.
- In-System / In-Circuit (Online): Connect external tools to the PCB to read the chip without desoldering.
- Methods: SoC debug interfaces (JTAG/SWD) or clamping directly to the storage chip pins (e.g., using a test clip).
- Tools: J-Link, USBDM, Bus Pirate, or custom harnesses.
- Internal Backup (Software): Gain shell access (e.g., via UART or partial exploit) and use system tools (dd, cat, nanddump) to dump the firmware partitions.
Tip: You don’t always need expensive programmers. For common protocols, a microcontroller (STM32, AVR) or a Raspberry Pi can often be repurposed as a dumper.
Architecture: How Embedded Devices Use FlashNOR Flash is similar to standard RAM: it supports random access. This enables XIP (eXecute In Place), allowing the CPU to fetch and execute instructions directly from the flash memory. This makes NOR ideal for storing the bootloader or BIOS, which must run immediately upon power-up.
NAND Flash, by contrast, does not support XIP. The CPU cannot execute code directly from NAND. Therefore, the very first stage of boot code cannot reside solely on NAND.
Historical Context (The “NOR-less” Shift): Early devices (like feature phones) used both NOR and NAND: NOR for the bootloader/kernel (XIP) and NAND for the filesystem (storage). Samsung later popularized the “NOR-less” concept. By embedding a small ROM inside the SoC capable of loading a bootloader from the first page of NAND into internal RAM, they eliminated the need for expensive NOR chips. This reduced cost and complexity, making NOR rare in modern high-volume consumer electronics like smartphones.
Managed Flash & FTL: NAND is susceptible to bit flips and bad blocks. It requires a complex software management layer called the FTL (Flash Translation Layer) to handle:
- Error Correction Codes (ECC)
- Bad Block Management
- Wear Leveling
- Garbage Collection
Depending on where this FTL resides, flash is categorized as:
- Raw Flash: The FTL is implemented in the OS driver (software).
- Managed Flash (eMMC, SD, UFS): The FTL is implemented in a hardware controller inside the storage package.
Implications for Extraction: When reading Raw NAND, you get the raw data including bit errors and OOB (Out-Of-Band) metadata. You must manually handle ECC algorithms (e.g., Hamming, BCH) and descrambling to reconstruct a valid binary. Since these algorithms are often vendor-specific and not standard, this is the most challenging part of raw firmware extraction.
Advanced topics on reconstructing firmware from raw dumps will be covered in future posts.
ReferencesNAND vs. NOR Flash Memory Technology Overview
Understanding Flash: Blocks, Pages and Program / Erases
UEFI Blog 杂谈闪存二:NOR和NAND Flash
Firmware Extraction Series: UBI Filesystem Extraction and Repacking
I originally wrote this post last year but accidentally set the GitHub repository to private and lost the README. After re-uploading, the context felt slightly dated, but the technical content remains relevant.
UBI (Unsorted Block Images) is a volume management system for raw flash devices designed by IBM. It can manage multiple logical volumes on a single physical device and supports wear leveling. It is widely used in embedded devices.
Speaking of raw flash, we should first explain what MTD (Memory Technology Device) is. MTD is a Linux subsystem used to access memory devices (especially flash devices). It serves as an abstraction layer between hardware and filesystems. Taking NAND flash as an example, MTD encapsulates NAND operations and provides abstract interfaces to upper-layer filesystem drivers. An MTD device consists of eraseblocks. The MTD driver provides read, write, and erase operations—but before modifying any block, you must erase it first.
UBI LayoutUBI shares similarities with LVM (Logical Volume Management). While LVM maps logical sectors to physical sectors, UBI maps Logical Erase Blocks (LEBs) to Physical Erase Blocks (PEBs). Fundamentally, UBI operates at the eraseblock level.
At the beginning of each UBI block (excluding bad blocks), there are two 64-byte headers:
- EC header (erase counter header), which contains per-PEB information (VID header offset, data offset).
- VID header (volume identifier header), which contains the volume ID and the PEB number corresponding to a given LEB.
In the Linux source tree under linux/drivers/mtd/ubi/, ubi-media.h defines both the EC header and VID header.
struct ubi_ec_hdr { __be32 magic; // UBI# __u8 version; // 01 __u8 padding1[3]; __be64 ec; /* Warning: the current limit is 31-bit anyway! */ __be32 vid_hdr_offset; // VID Header 的偏移 __be32 data_offset; // 数据的偏移 __be32 image_seq; // 物理块序号 __u8 padding2[32]; __be32 hdr_crc; // CRC32 } __packed; /* * UBI volume type constants. * * @UBI_DYNAMIC_VOLUME: dynamic volume * @UBI_STATIC_VOLUME: static volume */ enum { UBI_DYNAMIC_VOLUME = 3, UBI_STATIC_VOLUME = 4, }; struct ubi_vid_hdr { __be32 magic; // UBI! __u8 version; // 1 __u8 vol_type; // 一般是UBI_DYNAMIC_VOLUME __u8 copy_flag; // 是否从另一个物理块拷贝过来的(wear-leveling) __u8 compat; // 卷兼容性 __be32 vol_id; // 卷ID __be32 lnum; // LEB编号 __u8 padding1[4]; __be32 data_size; // 数据大小 __be32 used_ebs; // 用户LEB数量 __be32 data_pad; __be32 data_crc; __u8 padding2[4]; __be64 sqnum; // 序号 __u8 padding3[12]; __be32 hdr_crc; // CRC32 } __packed;The volume with ID UBI_INTERNAL_VOL_START is dedicated to storing volume table records.
#define UBI_INTERNAL_VOL_START (0x7FFFFFFF - 4096)It contains the volume name:
struct ubi_vtbl_record { __be32 reserved_pebs; __be32 alignment; __be32 data_pad; __u8 vol_type; __u8 upd_marker; __be16 name_len; // 卷名长度 __u8 name[UBI_VOL_NAME_MAX+1]; // 卷名 __u8 flags; __u8 padding[23]; __be32 crc; // CRC32 } __packed;On a typical MTD device, the initial sectors are reserved for the bootloader, while the subsequent regions are allocated for UBI. The figure below provides a simplified example; in practice, multiple UBI volumes and other partitions may be interleaved. UBI employs a mechanism called “fastmap” to map LEBs onto non-contiguous PEBs, providing an abstraction layer for UBIFS.
Mounting UBIFSMTD provides user-space tools for directly operating on UBI: mtd-utils
http://git.infradead.org/mtd-utils.git
- ubinfo - provides information about UBI devices and volumes found in the system;
- ubiattach - attaches MTD devices (which describe raw flash) to UBI and creates corresponding UBI devices;
- ubidetach - detaches MTD devices from UBI devices (the opposite to what ubiattach does);
- ubimkvol - creates UBI volumes on UBI devices;
- ubirmvol - removes UBI volumes from UBI devices;
- ubiblock - manages block interfaces for UBI volumes. See here for more information;
- ubiupdatevol - updates UBI volumes; this tool uses the UBI volume update feature which leaves the volume in “corrupted” state if the update was interrupted; additionally, this tool may be used to wipe out UBI volumes;
- ubicrc32 - calculates CRC-32 checksum of a file with the same initial seed as UBI would use;
- ubinize - generates UBI images;
- ubiformat - formats empty flash, erases flash and preserves erase counters, flashes UBI images to MTD devices;
- mtdinfo - reports information about MTD devices found in the system.
While the above tools operate on UBI, standard PCs lack native MTD devices. To analyze a raw flash firmware dump from an embedded device on a PC, you must simulate an MTD device. The most common tool for this is NANDSim.
- mtdram which simulates NOR flash in RAM;
- nandsim which simulates NAND flash in RAM;
- block2mtd which simulates NOR flash on top of a block device;
First, examine NANDSim’s parameters. Given the large number of options, correct configuration is key.
$ modinfo nandsim filename: /lib/modules/4.18.10-arch1-1-ARCH/kernel/drivers/mtd/nand/raw/nandsim.ko.xz description: The NAND flash simulator author: Artem B. Bityuckiy license: GPL srcversion: D2FD00330F9BE30A9B28365 depends: mtd,nand retpoline: Y intree: Y name: nandsim vermagic: 4.18.10-arch1-1-ARCH SMP preempt mod_unload modversions sig_id: PKCS#7 signer: sig_key: sig_hashalgo: md4 signature: parm: id_bytes:The ID bytes returned by NAND Flash 'read ID' command (array of byte) parm: first_id_byte:The first byte returned by NAND Flash 'read ID' command (manufacturer ID) (obsolete) (byte) parm: second_id_byte:The second byte returned by NAND Flash 'read ID' command (chip ID) (obsolete) (byte) parm: third_id_byte:The third byte returned by NAND Flash 'read ID' command (obsolete) (byte) parm: fourth_id_byte:The fourth byte returned by NAND Flash 'read ID' command (obsolete) (byte) parm: access_delay:Initial page access delay (microseconds) (uint) parm: programm_delay:Page programm delay (microseconds (uint) parm: erase_delay:Sector erase delay (milliseconds) (uint) parm: output_cycle:Word output (from flash) time (nanoseconds) (uint) parm: input_cycle:Word input (to flash) time (nanoseconds) (uint) parm: bus_width:Chip's bus width (8- or 16-bit) (uint) parm: do_delays:Simulate NAND delays using busy-waits if not zero (uint) parm: log:Perform logging if not zero (uint) parm: dbg:Output debug information if not zero (uint) parm: parts:Partition sizes (in erase blocks) separated by commas (array of ulong) parm: badblocks:Erase blocks that are initially marked bad, separated by commas (charp) parm: weakblocks:Weak erase blocks [: remaining erase cycles (defaults to 3)] separated by commas e.g. 113:2 means eb 113 can be erased only twice before failing (charp) parm: weakpages:Weak pages [: maximum writes (defaults to 3)] separated by commas e.g. 1401:2 means page 1401 can be written only twice before failing (charp) parm: bitflips:Maximum number of random bit flips per page (zero by default) (uint) parm: gravepages:Pages that lose data [: maximum reads (defaults to 3)] separated by commas e.g. 1401:2 means page 1401 can be read only twice before failing (charp) parm: overridesize:Specifies the NAND Flash size overriding the ID bytes. The size is specified in erase blocks and as the exponent of a power of two e.g. 5 means a size of 32 erase blocks (uint) parm: cache_file:File to use to cache nand pages instead of memory (charp) parm: bbt:0 OOB, 1 BBT with marker in OOB, 2 BBT with marker in data area (uint) parm: bch:Enable BCH ecc and set how many bits should be correctable in 512-byte blocks (uint)To understand NANDSim’s implementation, one can examine the kernel driver source code. Briefly, nandsim.c calls nand_scan_ident in nand_base.c. During nand_detect, a Read ID operation is performed: nand_readid_op sends 0x90,0x00 to the NAND device. Subsequently, nand_get_manufacturer matches the manufacturer ID. Finally, nand_scan_tail initializes the NAND chip and sets the appropriate properties.
The NAND chip datasheet specifies the ID bytes in detail.
Therefore, we must set the ID correctly. This allows the driver to automatically configure the capacity, page size, and other parameters. For NANDSim, only the first four ID parameters are required.
static u_char id_bytes[8] = { [0] = CONFIG_NANDSIM_FIRST_ID_BYTE, [1] = CONFIG_NANDSIM_SECOND_ID_BYTE, [2] = CONFIG_NANDSIM_THIRD_ID_BYTE, [3] = CONFIG_NANDSIM_FOURTH_ID_BYTE, [4 ... 7] = 0xFF, };If you need to tune the NAND simulation parameters, use the datasheet table to choose appropriate values.
Typically, an embedded device stores the bootloader and other partitions on the same chip as the system partition. Consequently, partitioning NANDSim is necessary. In this example, the eraseblock size is 128KB (0x20000).
Write a script to locate UBI regions in the dump:
#!/usr/bin/env python3 import sys import binascii import struct if len(sys.argv) < 1: print("Usage: find_ubi_header.py NAND.bin") sys.exit(1) raw_file_path = sys.argv[1] ubi_header = b'UBI#' out_of_ubi = True try: with open(raw_file_path, 'rb') as raw_file: rawbin = raw_file.read() for x in range(0, len(rawbin), 0x20000): magic = rawbin[x:x+4] if magic == ubi_header: if out_of_ubi: out_of_ubi = False print("\nUBI offset start:", hex(x)) else: if not out_of_ubi: print("UBI offset stop:", hex(x), "\n") out_of_ubi = True raw_file.close() except Exception as e: print(e) $ python find_ubi_header.py NAND.bin UBI offset start: 0x2e60000 UBI offset stop: 0x6900000 UBI offset start: 0x7700000 UBI offset stop: 0x81c0000 UBI offset start: 0x8200000 UBI offset stop: 0x20000000512MB = 4096 * 128 KB, so this chip has 4K blocks.
PN SA EA EC xxx 0x00000000 0x02E60000 371 ubi1 0x02E60000 0x06900000 469 foo 0x06900000 0x069C0000 6 recovery 0x069C0000 0x07700000 106 ubi2 0x07700000 0x081C0000 86 sec 0x081C0000 0x08200000 2 ubi3 0x08200000 0x20000000 3056Load the MTD modules and the NANDSim module:
sudo modprobe mtd sudo modprobe mtdblock sudo modprobe nandsim first_id_byte=0x2c second_id_byte=0xac third_id_byte=0x90 fourth_id_byte=0x15 parts=371,469,6,106,86,2,3056Verify the MTD device information to ensure the partitions were created successfully.
$ mtdinfo -a Count of MTD devices: 8 Present MTD devices: mtd0, mtd1, mtd2, mtd3, mtd4, mtd5, mtd6, mtd7 Sysfs interface supported: yes mtd0 Name: NAND 512MiB 1,8V 8-bit Type: nand Eraseblock size: 131072 bytes, 128.0 KiB Amount of eraseblocks: 4096 (536870912 bytes, 512.0 MiB) Minimum input/output unit size: 2048 bytes Sub-page size: 512 bytes OOB size: 64 bytes Character device major/minor: 90:0 Bad blocks are allowed: true Device is writable: true mtd1 Name: NAND simulator partition 0 Type: nand Eraseblock size: 131072 bytes, 128.0 KiB Amount of eraseblocks: 371 (48627712 bytes, 46.4 MiB) Minimum input/output unit size: 2048 bytes Sub-page size: 512 bytes OOB size: 64 bytes Character device major/minor: 90:2 Bad blocks are allowed: true Device is writable: true mtd2 Name: NAND simulator partition 1 Type: nand Eraseblock size: 131072 bytes, 128.0 KiB Amount of eraseblocks: 469 (61472768 bytes, 58.6 MiB) Minimum input/output unit size: 2048 bytes Sub-page size: 512 bytes OOB size: 64 bytes Character device major/minor: 90:4 Bad blocks are allowed: true Device is writable: true mtd3 Name: NAND simulator partition 2 Type: nand Eraseblock size: 131072 bytes, 128.0 KiB Amount of eraseblocks: 6 (786432 bytes, 768.0 KiB) Minimum input/output unit size: 2048 bytes Sub-page size: 512 bytes OOB size: 64 bytes Character device major/minor: 90:6 Bad blocks are allowed: true Device is writable: true mtd4 Name: NAND simulator partition 3 Type: nand Eraseblock size: 131072 bytes, 128.0 KiB Amount of eraseblocks: 106 (13893632 bytes, 13.2 MiB) Minimum input/output unit size: 2048 bytes Sub-page size: 512 bytes OOB size: 64 bytes Character device major/minor: 90:8 Bad blocks are allowed: true Device is writable: true mtd5 Name: NAND simulator partition 4 Type: nand Eraseblock size: 131072 bytes, 128.0 KiB Amount of eraseblocks: 86 (11272192 bytes, 10.8 MiB) Minimum input/output unit size: 2048 bytes Sub-page size: 512 bytes OOB size: 64 bytes Character device major/minor: 90:10 Bad blocks are allowed: true Device is writable: true mtd6 Name: NAND simulator partition 5 Type: nand Eraseblock size: 131072 bytes, 128.0 KiB Amount of eraseblocks: 2 (262144 bytes, 256.0 KiB) Minimum input/output unit size: 2048 bytes Sub-page size: 512 bytes OOB size: 64 bytes Character device major/minor: 90:12 Bad blocks are allowed: true Device is writable: true mtd7 Name: NAND simulator partition 6 Type: nand Eraseblock size: 131072 bytes, 128.0 KiB Amount of eraseblocks: 3056 (400556032 bytes, 382.0 MiB) Minimum input/output unit size: 2048 bytes Sub-page size: 512 bytes OOB size: 64 bytes Character device major/minor: 90:14 Bad blocks are allowed: true Device is writable: trueYou can also check dmesg for detailed load info, including chip and partition info.
$ dmesg [13202.334289] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xac [13202.334290] nand: Micron NAND 512MiB 1,8V 8-bit [13202.334291] nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64 [13202.334299] flash size: 512 MiB [13202.334299] page size: 2048 bytes [13202.334300] OOB area size: 64 bytes [13202.334300] sector size: 128 KiB [13202.334301] pages number: 262144 [13202.334301] pages per sector: 64 [13202.334302] bus width: 8 [13202.334302] bits in sector size: 17 [13202.334302] bits in page size: 11 [13202.334303] bits in OOB size: 6 [13202.334304] flash size with OOB: 540672 KiB [13202.334304] page address bytes: 5 [13202.334304] sector address bytes: 3 [13202.334305] options: 0x8 [13202.334779] Scanning device for bad blocks [13202.358806] Creating 7 MTD partitions on "NAND 512MiB 1,8V 8-bit": [13202.358810] 0x000000000000-0x000002e60000 : "NAND simulator partition 0" [13202.360129] 0x000002e60000-0x000006900000 : "NAND simulator partition 1" [13202.360835] 0x000006900000-0x0000069c0000 : "NAND simulator partition 2" [13202.361180] 0x0000069c0000-0x000007700000 : "NAND simulator partition 3" [13202.363506] 0x000007700000-0x0000081c0000 : "NAND simulator partition 4" [13202.365146] 0x0000081c0000-0x000008200000 : "NAND simulator partition 5" [13202.366440] 0x000008200000-0x000020000000 : "NAND simulator partition 6"You can also view the MTD partition table via:
$ sudo cat /proc/mtd dev: size erasesize name mtd0: 20000000 00020000 "NAND 512MiB 1,8V 8-bit" mtd1: 02e60000 00020000 "NAND simulator partition 0" mtd2: 03aa0000 00020000 "NAND simulator partition 1" mtd3: 000c0000 00020000 "NAND simulator partition 2" mtd4: 00d40000 00020000 "NAND simulator partition 3" mtd5: 00ac0000 00020000 "NAND simulator partition 4" mtd6: 00040000 00020000 "NAND simulator partition 5" mtd7: 17e00000 00020000 "NAND simulator partition 6"mtd0 represents the entire MTD device. Write the extracted firmware into this device; since the simulation runs in RAM, the operation is extremely fast.
sudo dd if=NAND.bin of=/dev/mtd0 bs=512M count=1Examining the ubi module parameters reveals an mtd parameter. However, using this may fail if the default VID header offset (512) does not match the target image.
sudo modprobe ubi mtd=0 $ dmesg [38418.429799] ubi0: attaching mtd5 [38418.429924] ubi0 error: validate_ec_hdr [ubi]: bad VID header offset 2048, expected 512 [38418.429937] ubi0 error: validate_ec_hdr [ubi]: bad EC header [38418.429944] Erase counter header dump: [38418.429946] magic 0x55424923 [38418.429948] version 1 [38418.429950] ec 5 [38418.429952] vid_hdr_offset 2048 [38418.429953] data_offset 4096 [38418.429955] image_seq 34870392 [38418.429957] hdr_crc 0x11db9c17Instead, load the ubi module first, then use ubiattach from mtd-utils to specify the parameters explicitly:
sudo modprobe ubi sudo ubiattach /dev/ubi_ctrl -m 2 -O 2048You should then see attach success messages:
[43880.484837] ubi0: default fastmap pool size: 20 [43880.484841] ubi0: default fastmap WL pool size: 10 [43880.484843] ubi0: attaching mtd2 [43880.486802] ubi0: attached by fastmap [43880.486806] ubi0: fastmap pool size: 20 [43880.486808] ubi0: fastmap WL pool size: 10 [43880.491518] ubi0: attached mtd2 (name "NAND simulator partition 1", size 58 MiB) [43880.491521] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes [43880.491523] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 512 [43880.491525] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096 [43880.491527] ubi0: good PEBs: 469, bad PEBs: 0, corrupted PEBs: 0 [43880.491529] ubi0: user volume: 1, internal volumes: 1, max. volumes count: 128 [43880.491532] ubi0: max/mean erase counter: 14/5, WL threshold: 4096, image sequence number: 1328192 [43880.491534] ubi0: available PEBs: 0, total reserved PEBs: 469, PEBs reserved for bad PEB handling: 80 [43880.491617] ubi0: background thread "ubi_bgt0d" started, PID 25777Next, mount the UBIFS volume to access the filesystem contents.
$ mkdir /tmp/modem $ sudo mount -t ubifs ubi0_0 /tmp/modem $ ls /tmp/image bdwlan30.bin mba.b03 mba.mdt modem.b03 modem.b08 modem.b12 modem.b16 modem.b22 otp30.bin mba.b00 mba.b04 modem.b00 modem.b05 modem.b09 modem.b13 modem.b19 modem.b23 qwlan30.bin mba.b01 mba.b05 modem.b01 modem.b06 modem.b10 modem.b14 modem.b20 modem.b24 utf30.bin mba.b02 mba.mbn modem.b02 modem.b07 modem.b11 modem.b15 modem.b21 modem.mdtIn some cases, SquashFS runs on top of UBI, which causes standard UBIFS mounts to fail:
[ 214.800087] UBIFS error (ubi0:0 pid 3848): ubifs_read_node [ubifs]: bad node type (1 but expected 6) [ 214.800093] UBIFS error (ubi0:0 pid 3848): ubifs_read_node [ubifs]: bad node at LEB 0:0, LEB mapping status 1 [ 214.800094] Not a node, first 24 bytes: [ 214.800095] 00000000: 68 73 71 73 46 0c 00 00 5a 9c 25 5d 00 00 02 00 ac 00 00 00 01 00 11 00 hsqsFThe hsqs magic bytes identify this as a SquashFS image. It can be extracted directly as SquashFS:
sudo dd if=/dev/ubi0_0 of=./ubi0_0 unsquashfs ./ubi0_0Detach/unload:
sudo umount MOUNTED_DIR sudo ubidetach /dev/ubi_ctrl -m 0 sudo modprobe -r ubi sudo modprobe -r nandsim Reading with UBI ReaderInstall via pip (or download): https://github.com/jrspruitt/ubi_reader
sudo pip install ubi_readerFirst, check whether it correctly identifies UBI metadata:
ubireader_display_info [options] path/to/fileExtract all files (but it will fail if another filesystem is present):
ubireader_extract_files [options] path/to/fileIt is recommended to first restore PEBs to LEBs before analyzing individual volumes:
ubireader_extract_images [options] path/to/file Repacking UBIAfter mounting UBIFS, modification and repacking might be required. Direct use of dd is not suitable for this task.
First, pay attention to the output from ubiattach—it prints LEB information:
$ sudo ubiattach /dev/ubi_ctrl -m 7 -O 2048 UBI device number 0, total 240 LEBs (30474240 bytes, 29.1 MiB), available 0 LEBs (0 bytes), LEB size 126976 bytes (124.0 KiB)There are 240 LEBs in total, with each LEB being 126976 bytes. These parameters must be passed to mkfs.ubifs to build the UBIFS image:
# mtd5 sudo mkfs.ubifs -m 2048 -e 126976 -c 240 -F -r ./UBI_1 rootfs.img # mtd9 sudo mkfs.ubifs -m 2048 -e 126976 -c 240 -F -r ./UBI_2 rootfs.img sudo mkfs.ubifs -m 2048 -e 126976 -c 240 -R 1 -x lzo -r ./UBI_1 rootfs.img sudo mkfs.ubifs -m 2048 -e 126976 -c 240 -x lzo -r ./rootfs rootfs.imgCreate ubi_config.ini:
vi ubi_config.iniThe vol_size must match the image size. Additionally, the file must end with an empty line to avoid the following error: ubinize: error!: cannot load the input ini file "ubi_config.ini"
[rootfs] mode=ubi image=rootfs.img vol_id=0 vol_size=9904128 vol_type=dynamic vol_name=rootfs vol_alignment=1 vol_flags=autoresizeFinally, use ubinize to generate the UBI image:
sudo ubinize -o rootfs.ubi -p 131072 -m 2048 -s 512 -e 2 -Q 0 -O 2048 -x1 ubi_config.iniWhere:
- -e: Number of eraseblocks (default is 0). This can be quickly verified with binwalk.
- -Q: Image sequence number, viewable with ubi_display_info.
- -x: UBI version (default is 1).
- -s: Sub-page size. Not all NAND flash supports sub-pages. Typically, SLC NAND with 2048-byte pages uses 4 sub-pages of 512 bytes, whereas MLC usually does not have sub-pages.
- -m: Page size.
- -p: Physical eraseblock size. A physical block usually consists of 64 pages; refer to the NAND datasheet for specifics.
Below is an example of flashing rootfs.ubi to mtd7:
sudo ubinize -v -o rootfs.ubi -p 131072 -m 2048 -s 512 -O 2048 ubi_config.ini sudo ubiformat /dev/mtd7 -O 2048 -s 512 -f rootfs.ubiFor SquashFS filesystems, building UBIFS is unnecessary. After modifying the contents, simply repack SquashFS using mksquashfs. Ensure that ownership and permissions are correct (e.g., if the target system runs as root, pack as root). Then, use ubinize to wrap it into a UBI image.
sudo mksquashfs ./squashfs-root/* rootfs.squashfs Reference