ZIP structure, DISK

I. Overview

Let's explore the structure of a ZIP file. We will examine how the header is structured and how the information associated with each header is organized. Disks can be categorized as primary and secondary storage devices. We will briefly review the working principles and structure of disks (MBR, VBR), and understand how they function in real-world computers. Finally, we will compare and contrast HDDs and SSDs, focusing on the differences in file reading, writing, deletion, and copying, particularly in terms of speed. As a practical exercise, we will use FTK Imager to perform image dumping to analyze MBR and VBR through HxD (this exercise will be covered in the next post).

II. ZIP Structure

Analysis of the structure of a ZIP file.

I'd like to share a blog post that provides information about the structure of ZIP files. It's quite detailed, so I recommend checking it out.

The structure of a ZIP file is as follows: Local File Header, Central Directory File HeaderEach compressed individual file contains this information at the end. End of Central Directory Record HeaderThis concludes the explanation. Now, let's examine the components of each header.

1. Local File Header

The Local File Header is structured as follows, and for more detailed information, please refer to the blog post. The orange-colored Signature, Version, and Flags are fixed in bytes. However, the green-colored File Name, Extra Field, and Data sections can vary, for example, in the case of Notepad, the length of the title and the amount of content can differ significantly.

2. Central Directory File Header

The Central Directory File Header is structured as follows, with the Local File Header providing more detailed data information. It includes information such as compression size, file description, version, and modification date.

3. End of Central Directory Header

The Local File Header and Central Directory File Header can exist multiple times, corresponding to the number of compressed files. However, the End of Central Directory Header, as its name suggests, contains the overall information for the entire compressed ZIP file. Finally, a single header.This section, specifically the 0xB0, 0xC0 part, contains information about the total number or size of the Central Directory, as well as details about the ZIP file itself.

※ This refers to practical training ZIP I've written code in Python to automatically import files as binary data and extract the headers and their properties. While the code is quite sophisticated, I'm hesitant to share it because I believe that actually trying it out is the best way to learn the underlying structure. In reality, you can modify the extracted hexadecimal values, such as changing the modification date, title, or content. However, it's something that can be done for fun.

※ To clarify, this file structure is used in almost all common file formats. You may have encountered file extensions like .hwpx (for Korean files) and .xlsx (for Excel). These are examples of compressed file formats.

Ⅲ. Disk

DiskThe types are: Primary storageWow Auxiliary storage deviceThere is data available. Previously, the data was SequentialFurthermore, finding the intermediate data proved to be extremely difficult, as it required going back to the original data. In addition, the data had non-persistent characteristics, which meant that modern hard drives... "Random Access Method of Accounting Control" By employing this technique, we were able to improve it. Periodic storage devices typically... RAM (Random Access Memory)Which is also known as Volatile dataThis is where the data is stored. As mentioned in a previous post, RAM is volatile memory, meaning the data is lost when the power is turned off. However, flash memory (like USB drives) is non-volatile, so the data remains even when the power is off. This is a key advantage, as it allows us to keep all our programs and documents on our computer, even when we restart it. Non-volatile storage deviceUnlike RAM (Random Access Memory), which is volatile. Non-volatile dataWhile it can store data, it is generally slower and referred to as a secondary storage device.

There's also something called a "Disc," which is different from a "Disk." Audio CD, CD-ROM (Read Only Memory), DVD-ROM (Read Only Memory), DVD-RAM, etc. This refers to optical media. Some are designed for reading only and can be disconnected from a computer at any time. There are also discs that can store content (such as music or video) and can be reused multiple times. Think of a CD for easier understanding. Just as a network interface is needed when working with networks in VMware, a similar interface is needed for discs. IDE (Integrated Drive Electronics) Or ATA (Advanced Technology Attachment)However, in the sense of "parallel". PATA (Parallel Advanced Technology Attachment)This is called...

PATA was a widely used method in the past, but now it's being replaced by methods that improve the speed of existing IDEs. SATA (Serial Advanced Technology Attachment)It uses SATA1, which offers a speed of 150mb per second, and SATA2, which offers a speed of 300mb per second. With SATA2, it's possible to connect multiple hard drives to a single port, similar to using a USB hub. You can connect the device while the PC is on, just like using a USB. Hot plug Additional features have also been added.

※ SCSI (Small Computer System Interface): It also utilizes a disk interface compatible with bank notes. With a transfer rate of 320 MB per second, it is significantly faster than typical hard drives. It is designed for use with servers or workstations (high-performance computers similar in performance to mini-computers, offering faster processing speeds than general-purpose PCs) and requires a separate expansion card. SAS (Serial Attached SCSI - a type of storage interface commonly used in servers and other large computers)This is evolving and being replaced by something else.

When examining the boot process, it can generally be observed that:

You may have heard a "beep" sound when turning on a computer. This is POST (Power-On Self Test)This process involves checking whether the hardware devices are receiving adequate power. Once the overall hardware check is complete, we move on to… BIOS You will see the screen. If you have experience with formatting, you may have already tried changing the settings here. The BIOS allows you to manage hardware settings and make some... Read-only memory (ROM)This option allows you to choose a boot disk or overclock.

Next, MBR (Master Boot Record)This process is extremely important. If it fails, the system will not boot, as the BIOS relies on reading the MBR's boot sector to identify bootable devices. If you examine the entire disk, which is 512 bytes in size, you will first see the 0th sector. Information for a total of 4 partitions: the main partition and three auxiliary partitions.Each partition contains 16 bytes of information, with each partition holding information from 446 bytes up to the last 2 bytes, excluding the last 2 bytes.

※ Here SectorWhen examining the disk structure, it's important to note that the disk is divided into multiple sectors for data storage and management. The last 4 bytes of information represent the total number of sectors in a specific partition, rather than the entire disk. And a collection of multiple sectors ClusterThe larger the cluster, the faster it is. This is because reading and writing data in clusters is inherently faster than reading and writing data sector by sector. However, as clusters become larger, the amount of unused space increases, leading to significant waste.

Since this explanation might be difficult to understand, please refer to the image for clarity. MBR (Master Boot Record) can be up to 446 bytes. Boot CodeHaving that, and then... Partition TableThere are a total of four, ranging from 446 to 510 bytes. The last one is 2 bytes long. SignatureThis refers to the partition that is recorded in the MBR (Master Boot Record). This partition is the primary partition, which can be used to boot the system. A disk can have one or more partitions. When you first install a disk, you will be prompted to create partitions, and you will see this option during the formatting process. You may have divided your C and D drives, with the D drive often being used for backups. Of course, both drives could also be separate disks.

※ Because the MBR (Master Boot Record) contains the boot code, and there is no backup, recovery is extremely difficult if the boot code becomes corrupted.

In the past HDD (Hard Disk Drive)I used to write a lot, but these days... SSD (Solid State Drive)We will use this method. HDDs utilize the MBR method described above, while SSDs use GPT (GUID Partition Table) GPT offers several advantages over MBR in terms of partition management. Unlike MBR, which was limited to a maximum of four partitions and had a 2TB capacity limit, GPT has no such restrictions, allowing for greater flexibility and convenience in managing partitions.

※ Logic diskWow Physical disc There's a concept of "logical disk," which refers to a partition on a disk, and a "physical disk," which refers to the actual hard drive hardware, such as an HDD or SSD.

After MBR and GPT handle the partitioning, the file system is configured, as shown in the image. VBR (Volume Boot Record)Okay. Master File TableThis allows you to read and write files. It manages where data is stored on different partitions, and the actual data is stored as shown in the image. Data AreaThis VBR (Volume Boot Record) is stored and managed in conjunction with the Master File Table. While it allows for easy restoration due to backups, it can also be problematic if the system fails to boot. You might encounter this VBR when Windows prompts you to choose a formatting method. FAT, NTFS Essentially, this file system is FAT. While FAT had a limitation of a maximum file size of 4GB and a 32-bit length restriction, NTFS overcomes these limitations, eliminating the 4GB limit and generally offering improvements, making it the most widely used.

※ Of course, there is also a newer version of FAT called FAT32. In addition to FAT32, it includes a reserved area after VBR (for storing boot sectors, basic file system information, etc.) and records which files are assigned to FAT1 and FAT2 (which have a mirroring relationship and the same structure). Following that is the MFT (Master File Table), which stores file information.

NTFS (Next Technology File System)Wow FAT (File Allocation Table)The Master File Table is responsible for reading, writing, and deleting files. Typically, when a file is created, MBRAccess the partition and obtain information about it. VBR (NTFS, FAT)Accessing VBR. MFT (Master File Table)Based on this, the system calculates the available space in the Data Area and finds a new file to save. Then, it stores information about the available space in the Data Area in the MFT (Master File Table) and finally saves the file. You may notice a difference in speed when copying or moving files.

※ When using NTFS, if the file size is small, the file content is not stored in the Data Area but is instead recorded in the Master File Table (MFT).

The reason for this difference is:

You should be able to understand this by looking at the image. Within the same partitionWhen copying files, you need to create a new file in the Data Area and update the MFT with information (file name, date, size, storage location, etc.). However, when moving files, you only need to update the information in the MFT, making it faster. Another partitionWhen copying data from a CD/DVD, the process is slower because it requires modifying the MFT (Master File Table) and writing new file information to the Data Area, just like copying within the same partition. The same applies to moving data. However, in general Deleteis

Regarding the use of MTF Delete only the link information in the Data Area.Because it's straightforward, it's very fast. What happens if you use the overwrite function?

Overwriting is a standard method, but it doesn't work for recovery because it corrupts the data. When you modify the Data Area of the existing file, the MFT header information becomes invalid. This means the information in the MFT is replaced with the information from the overwritten data. As a result, the original file's information is lost. The difference between HDDs and SSDs lies here: While HDDs can be overwritten, SSDs generally cannot. Trimming techniques However, in the past, this feature was exploited to make data recovery difficult by intentionally overwriting data on the hard drive, taking advantage of the fact that it was not possible to restore overwritten files. RandomThe "Trim" technique, which operates using [specific technology/method], Manage file data that is not regularly used.In practice, if you only modify with 0 and 1, it's sufficient. However, if you do that, the existing data will be considered deleted. Because TRIM identifies and sets the status as "invalid" at the moment the link between the MFT and the data is broken, and because it's very fast, overwriting is difficult.

※ While garbage collection was present in previous versions of Windows, such as XP, Windows 7 introduced the "Trim" feature.

※ When copying files, the creation time may be slower than the modification time. This occurs when the files are transferred via the internet or USB, meaning they are being moved between different disks rather than within the same partition. When files are received from the internet, the "Zone.Identifier" will be displayed (when checked using the "dir" command in CMD). As for USB drives, they are recognized and drivers are installed immediately upon connection using plug-and-play.

Another aspect related to speed is FragmentationIt's a concept.

As shown in the image, older hard drives stored file data in separate partitions. Therefore, to retrieve a file, Link(If you've learned a programming language, thinking of linked lists will help you understand this.) We had to load and read the data from each file one by one. Since reading a single file is naturally faster than reading multiple files separately, we used to... Collection of disk fragmentsThey would have had to do a lot of work. However, nowadays, because files are stored sequentially in a single file rather than being fragmented, it is much faster. Another concept related to speed is DISK Nand Shape and NOR It has a form. "Nand" refers to a type of semiconductor cell. SeriesBecause it reads data sequentially, it is slower than NOR flash memory. However, it is easier to increase its capacity without needing to remember the address of each cell, and it offers faster write speeds. This NAND-based disk USB or SSDNOR is a logic gate that outputs a high signal if either the A or B input is high, but not if both are high. ParallelWhile it offers fast random access for reading, its writing speed is slow because it records data one cell at a time. This makes it suitable for applications where reading speed is prioritized, such as high-end devices and smartphones. Mobile devicesThis is because mobile devices tend to read data more often than they write it, especially when dealing with large amounts of data.

※ Just like the engine RPM of a car, a disc also has an RPM. RPM (Revolutions Per Minute)The faster the drive, the faster it operates. However, standard hard drives typically operate at 7,200 RPM, while those designed for mobile devices or laptops tend to be slower.

IV. Practical Exercises Related to Disks

First, the practical training involves The process of dumping data from a disk.In simple terms, the dump is created by converting the entire disk structure, as described earlier, into an image format, making it readable through the Imager program. Legally, if the original is damaged, it cannot be used as evidence. However, during a forensic investigation, the original cannot be used. Therefore, only the FTK Imager program, distributed by the forensic society, is permitted. I plan to use this program to create a disk dump.

Regarding the parts related to the program:

FTK® Imager - Exterro

Please refer to this webpage.

When you run FTK Imager,

The screen will display something like this. Assuming the USB is connected, I will proceed with dumping the data from the USB drive.

Click on "Create Disk Image"

When asking about which drive to use, "Logical Drive" refers to a partition, and "Image File" refers to an existing disk dump image. Therefore, I will skip those and simply select a physical drive for you.

Once you have selected the appropriate blank disc device and clicked "Finish",

Here, you specify the image type using the "Add" function. The most commonly used method is the E01 format, which is used by Encase. You may need to research this further.

If you continue to pass it on and share it,

You will then specify the location to save the file and the file name. Here, "Image Fragment Size" indicates the size to which the image file will be divided when saving, and "Compression" refers to the level of compression. A higher value for compression will result in a smaller file size, but it may also slow down the process of creating the image file.

And when you click "Start," the process of generating the image file will begin.

Review

I felt like I was hitting a wall when I was learning about DISK. However, by writing posts for repeated practice and review, I'm starting to get a better understanding of the structure. I plan to incorporate more practice related to DISK in my next post!