The_Programmers_ByAble/ATA, NVMe, SATA, Floppy Disks and Optical Drives; Storing my own personal hell.md

8.1 KiB

ATA, NVMe, SATA, Floppy Disks and Optical Drives; Storing my own personal hell

A storage media would reasonably have two features Store or write Load or read depending on your favored terminology.

Alright let us learn about ATA first because it is alphabetically first!

ATA

IDE is used mostly interchangably with ATA but IDE is the physical transmission not the protocol name. Subtle but important difference here down at the bottom. ATA seems to have multiple ways of Storeing and Loading data. Oh joy!

SATA and PATA

SATA is a new standard for data transmission it stands for Serial AT Attachment All old ATA devices are retroactively named PATA and it stands for (Parallel) ATA

The vast majority of SATA devices also support AHCI. AHCI was developed by intel to help handle SATA devices

To determine if an ATA device is a SATA device you send a standard IDENTIFY command (as specified by P/ATA) to the drive (0xEC) it responds with an ERR and a status pair...

Ok thats insane but sure...

It also seems there are 2-3 different standards for Storeing and Loading data in SATA mode. Lets assume for my sake and yours, dear reader, that the API is identical for these modes.

Let us define a simple API to interact with these devices.

pub struct DiskIO {
    @documentation("The pointer is a pointer into the DISK addressing")
    read: function(pointer: u64, length: u64) -> Vector<u8>;

    @validator(data: data.len() > 0)
    @documentation("The pointer is a pointer into the DISK addressing, the Vector must be u8s and must atleast have one u8")
    write: function(pointer: u64, data: Vector<u8>) -> None;
}

Lets move on to the older more legacy saner(?) ATA PIO Mode

ATA PIO Mode

ATA is a hack on an older specification named ST506 (Side note: I WILL NOT be addressing this)

The only difference anyways is the controller being moved on board and one less cable.

When accessing a disk the cpu shorts a pin and sets which disk of the master/slave pair is being accessed

Once a command has been submitted you may poll the drive to check the status of the command. You MUST poll 15 times OR delay by 400 nanoseconds after a command is submitted for it to complete. If you send a command to a drive before 400ns are up you can destroy the controller on badly made drives or just mess up the previous command on better behaving more lieniant devices.

On some drives it is a requirement to flush the hardware cache after each write if this is not done you can create bad sectors until the drive is powercycled.

This is insane and should not be the case... A cache that is flushed after every write is not a cache that is worth using...

But! None-the-less we have a new concept called a hardware cache, and it has to sometimes be flushed by the driver.

Another API

pub struct DiskIO {
    @documentation("Set this to false any time we write")
    flushed_hw_cache: bool,

    @documentation("The pointer is a pointer into the DISK addressing")
    read: function(pointer: u64, length: u64) -> Vector<u8>;

    @validator(data: data.len() > 0)
    @documentation("The pointer is a pointer into the DISK addressing, the Vector must be u8s and must atleast have one u8")
    write: function(pointer: u64, data: Vector<u8>) -> None;
    @validator(flushed: false)
    @documentation("Only to be called if we have not flushed yet")
    flush_hw_cache: function(None) -> None;
}

Well that is less than ideal but atleast it gives us more control! EXCEPT on drives where we MUST flush after every write.

We will not be learning about addressing modes because I gave myself a headache looking at them.

Next!

ATAPI

ATAPI; Adding a lil SCSI to your ATA. Typically CD-ROM, CD-RW, DVD, or tape drive attatched to the ATA Bus use ATAPI.

This is included in drives that are ATA Version 6 or greater. We will not be adding an API to get the version as any file system interacting with the disk driver would presumably not need to know about the disk.

More magical methods of identification..

A new ATA Packet type.

And finally getting Disc Contents!

First we must ensure that the disk is readable.

This will be handled by the disk driver and will not be reported by the API as the device cannot be used by a filesystem driver if it is not readable.

secondly we check if the disk is writable.

This will be included in the API because CD-ROM media can be read only for example

Previously we assumed that disks had infinite storage because in the modern day disks are infinitly big compared to a CD-ROM. We will add in the api for getting size now. We will also add a free space for convenience

pub struct DiskIO {
    @documentation("Set this to false any time we write")
    flushed_hw_cache: bool,

    @validator(size > 0)
    @documentation("Size is in bytes and must be greater than zero")
    size: u64,
    @validator(size >= free_space)
    free_space: u64,

    @documentation("The pointer is a pointer into the DISK addressing")
    read: function(pointer: u64, length: u64) -> Vector<u8>;

    @validator(data: data.len() > 0)
    @documentation("The pointer is a pointer into the DISK addressing, the Vector must be u8s and must atleast have one u8")
    write: function(pointer: u64, data: Vector<u8>) -> None;
    @validator(flushed: false)
    @documentation("Only to be called if we have not flushed yet")
    flush_hw_cache: function(None) -> None;

    @documentation("Returns True if the disk is writeable.")
    is_writeable: function(None) -> Bool;

}

ATA/ATAPI using DMA

There is no noticable API change here but therre is a noticable speed up over previous modes.

ATA in x86 RealMode (BIOS)

This adds nothing interesting and is only useful if you rely on BIOS calls. We do not.


Intermission One Act One

Well that was hell! Luckily thats over with and we move onto the simple things.

NVMe's!

NVMes

NVMes use the PCI bus to communicate.

Lets just check the spec real quick...

oh 458 pages

...dear me... and thats only for the Base specification. What ever that means Lets not look at that actually and glance over at the OSDev wiki, my favorite misinformed shit-show forum wiki combo.

NVMes :(

NVMes have a few registers to communicate back and forth with the driver.

11 registers of which 2 are static so 9 registers

with these registers you create an admin queue to create submission buffers to feed commands to the NVMe drive

The NVMe controller may process commands in any order it likes. oh dear

so you must figure out which commands cannot yet be submitted do to dependence on data from other commands yourself and have a second submission buffer queue....

What the FUCK! Please give me back any of the ATA modes.

Floppy Disks

3 modes because of different chipsets

most commands are silent, ala no status updates when complete. Great! more active polling, of the data this time not the disk

floppy disk controllers are configured via 9 registers that are IO memory mapped

depending on if you read or write to them you are accessing different registers FUCK THIS PLEASE BE NORMAL FOR ONE MINUTE

In real hardware floppies are incredibly unrealiable so you are recommended to (write then read and compare) twice...

Intermission One Act Two

DiskIO.write(0, Vector("I am living in hell"));
// Oh and don't forget to flush the fucking cache
DiskIO.flush();

Optical Drives

Ahh back to the good old SCSI of the ATAPI minus the ATAPI packets alright.

So implement optical drives first

Random R+W operations on DVD and Bluray

Its just fucking magic ok.

SD Cards

SD Cards are predictably just as scuffed

Vendor Enhancements

Integrated Wi-Fi

No this cannot be real. We have to support Wi-Fi over DiskIO???????????? No We WILL not. Sorry the one person who needs this

Pre-loaded Content

What the actual fuck? Pre-loaded reserved content that is write protected...

oh and lets not forget DRM

Integrated USB Connector

This is just a built in sdcard reader usb combo

The sanest extension yet

Different colors

Similar to USB colors to identify card types

I actually like this and suggest others do this as well

Integrated display

Ok well thats it fools.