PST File Documentation

Summary

A PST (Outlook Personal Information Store File) is Microsoft Outlook’s local data file: a single binary database that holds an entire mailbox — folders of email plus contacts, calendar appointments, tasks and notes. Outlook opens one through File › Open & Export › Open Outlook Data File, and free viewers or readpst read it without Outlook. Its extension is .pst and its MIME type is application/vnd.ms-outlook.

Technical details

FeatureValue
Full nameOutlook Personal Information Store File (Personal Storage Table / Personal Folders File)
File extension.pst
MIME typeapplication/vnd.ms-outlook
Format typeProprietary binary database (whole-mailbox store)
DeveloperMicrosoft
Introduced1997 (Outlook 97, ANSI); Unicode PST since Outlook 2003
Standard / spec[MS-PST] Outlook Personal Folders File Format (published 2010)
Open standardPartial — proprietary format, but the spec is openly published
Byte orderLittle-endian
Magic number21 42 44 4E (ASCII !BDN) at offset 0
Version markerwVer at offset 0x000A: 14/15 = ANSI, ≥23 = Unicode
Client magicwMagicClient = 53 4D (ASCII SM) at offset 0x0008
ArchitectureLayered: NDB (node/block database) → LTP (lists, tables, properties) → Messaging
ANSI size limit2 GB (32-bit offsets, Outlook 97–2002)
Unicode size limit~50 GB default in current Outlook (64-bit byte indexes, configurable)
Block sizeData blocks up to 8192 bytes; B-tree pages are 512 bytes
Data encodingbCryptMethod: NONE, PERMUTE, or CYCLIC (obfuscation, not real encryption)
Password schemeCRC-32 hash in PidTagPstPassword — trivially bypassed
Integrity checkdwCRCPartial and dwCRCFull header CRCs; per-block signatures
Related extensions.ost, .msg, .olm, .eml, .mbox
Specification URLlearn.microsoft.com/openspecs/office_file_formats/ms-pst/
File signature (magic bytes)
21 42 44 4E

Offset 0, 4 bytes. In ASCII this reads ! B D N, the dwMagic field that opens every PST. The same !BDN header also begins an OST file, because the two share one on-disk format. The version word wVer at offset 0x000A then tells the two PST variants apart: 14/15 mark an ANSI PST (2 GB cap), a value of 23 or higher marks a Unicode PST. Immediately before it, at offset 0x0008, sits wMagicClient = 53 4D (ASCII SM), a second validation stamp.

What is a PST file?

PST stands for Personal Storage Table, also called the Personal Folders File. It is Microsoft Outlook’s local data-file format, introduced in 1997 with Outlook 97 and documented by Microsoft since 2010 in the open specification [MS-PST], the Outlook Personal Folders File Format. Unlike an EML or MSG file, which each hold a single message, a .pst is a whole mailbox in one file: every mail folder plus contacts, calendar appointments, tasks and notes. Its MIME type is application/vnd.ms-outlook.

Internally a PST is not an archive of individual files but a self-contained database. Outlook stores items as sets of tagged properties inside a b-tree store, and the file grows, splits and reclaims space much like a small on-disk database engine. The format is defined as three stacked layers, and everything below works through that stack: the NDB node/block database at the bottom, the LTP list/table/property layer in the middle, and the messaging layer of folders and messages on top.

The header at offset 0: dwMagic, wVer and the CRCs

Every PST opens with a fixed HEADER structure at absolute offset 0. It carries the signatures that identify the file, the version marker that selects ANSI or Unicode layout, two CRC checks, and the root reference that anchors the rest of the file. All multi-byte integers are little-endian.

Offset  Size  Field            Value / meaning
0x0000   4    dwMagic          21 42 44 4E  ("!BDN")  file signature
0x0004   4    dwCRCPartial     CRC-32 of 471 bytes from wMagicClient
0x0008   2    wMagicClient     53 4D  ("SM")  client signature
0x000A   2    wVer             14/15 = ANSI, >=23 = Unicode
0x000C   2    wVerClient       client format version (19 for this spec)
0x000E   1    bPlatformCreate  MUST be 0x01
0x000F   1    bPlatformAccess  MUST be 0x01
0x0010   8    dwReserved1/2    ignored, set to zero
0x0018   8    bidUnused        Unicode-only padding
0x0020   8    bidNextP         next page BID counter
0x0028   4    dwUnique         bumped on every header change
0x002C  128   rgnid[]          32 NIDs, last index per NID_TYPE
...      72   root             ROOT: BREFs to the NBT and BBT
...     128   rgbFM / rgbFP    deprecated maps, filled with 0xFF
...      1    bSentinel        MUST be 0x80
...      1    bCryptMethod     NONE / PERMUTE / CYCLIC
...      8    bidNextB         next block BID counter (Unicode)
...      4    dwCRCFull        CRC-32 of 516 bytes to bidNextB

The 4-byte dwMagic is the literal ASCII string !BDN. Two bytes at offset 0x0008, wMagicClient, hold SM (bytes 53 4D) as a second validation stamp. The 2-byte wVer at offset 0x000A is the field that matters most for a reader: 14 or 15 mean an ANSI PST, and any value of 23 or greater means a Unicode PST, which changes field sizes and the layout of the whole file. A value of 37 marks a file written by an Outlook that supports Windows Information Protection.

Two CRC-32 values guard the header. dwCRCPartial covers the 471 bytes that start at wMagicClient; the Unicode-only dwCRCFull covers the 516 bytes running from wMagicClient through bidNextB. dwUnique is a counter bumped on every header modification, which guarantees the CRCs differ after each change. The rgnid array is 128 bytes: 32 node IDs, one per NID type, each holding the last index allocated so the next new folder or message gets a fresh ID (normal folders start at 0x400, normal messages at 0x10000).

ANSI versus Unicode: two on-disk formats, one extension

The single .pst extension covers two genuinely different binary layouts, and wVer is what distinguishes them. The original ANSI PST (Outlook 97 through 2002) uses 32-bit byte indexes, which caps the file at 2 GB. As an ANSI PST approaches that ceiling it tends to corrupt, which is the classic cause of “Outlook won’t open my PST”. Block IDs and byte offsets are 4 bytes, and the root structure is 40 bytes.

The Unicode PST (Outlook 2003 and later) widens byte indexes to 64 bits, stores text as UTF-16, and raises the size ceiling far past 2 GB; current Outlook defaults to roughly a 50 GB cap that is configurable in the registry. In the Unicode layout the root structure grows to 72 bytes, block IDs and the IB byte index become 8 bytes, and the header gains bidUnused padding, a qwUnused field and the extra dwCRCFull check. Because the two formats differ in field widths throughout, a reader must branch on wVer before parsing anything past the first ten bytes.

The NDB layer: BREF, blocks, pages and the two b-trees

The lowest layer is the Node Database (NDB). It turns the flat file into a store of numbered nodes and their data, and everything above it is expressed in NDB terms. Four primitives do the work.

Messaging layer   folders, messages, attachments, named properties
      |           (built from Property Contexts and Table Contexts)
LTP layer         Lists, Tables, Properties: HN, BTH, PC, TC
      |           (structures laid out inside NDB data blocks)
NDB layer         header, BREF, blocks, pages, NBT (node b-tree),
      |           BBT (block b-tree)
raw file          little-endian bytes on disk

A BID is a block ID; an IB is a byte index, an absolute position in the file. A BREF pairs a BID with an IB, so a BREF is effectively “this block lives at this file offset”. Data is stored in blocks of up to 8192 bytes, each ending with a small trailer that records the block’s size, a signature and its own CRC. Larger items span multiple blocks, tied together by an intermediate XBLOCK that lists the child BIDs.

Two b-trees index the file, and each is built from 512-byte pages (BTPAGEs). The Node B-Tree (NBT) maps each node ID to the block that holds its data and, where present, its subnode tree. The Block B-Tree (BBT) maps each block ID to a BREF plus the block’s byte count and reference count, so the store knows where every block is and how many nodes point at it. The root structure in the header holds the BREFs that locate the roots of both trees. To read any item, Outlook looks its node up in the NBT, follows the reference into the BBT to find the block on disk, then hands the raw bytes to the layer above.

The LTP layer: heap-on-node, property contexts and table contexts

Raw blocks are just bytes. The Lists, Tables and Properties (LTP) layer imposes structure on them. It starts with the Heap-on-Node (HN), a small allocator carved inside a data block that hands out numbered fragments so several logical structures can share one block. On top of the heap sits the BTree-on-Heap (BTH), a compact b-tree whose nodes are heap allocations, giving fast keyed lookup without a separate on-disk tree.

Two higher constructs are built from those pieces. A Property Context (PC) is a BTH keyed by property tag: it maps each property (a 16-bit ID combined with a type, exactly as in MAPI) to its value, which is how a single message or folder stores its subject, dates, flags and body. A Table Context (TC) is a row-and-column table, used where many similar items must be listed, such as the contents of a folder. Each row in a TC is one item; columns are property tags; and large or variable-length values are stored in subnodes referenced from the row rather than inline. The messaging layer never touches raw blocks directly, it reads and writes PCs and TCs.

The messaging layer: folders, messages and attachments

The top Messaging layer gives the LTP structures their meaning as a mailbox. A folder is not a single object but a set of nodes: a Property Context for the folder’s own attributes (its display name, item counts), a Table Context listing its contained messages (the hierarchy and contents tables), and further tables for subfolders and associated data. The special root folder node is the entry point Outlook opens first.

A message is a Property Context holding its MAPI properties — sender, recipients, subject, timestamps, importance, the plain-text and HTML bodies — together with subnode Table Contexts for its recipient list and its attachment list. Each attachment is itself a sub-object with its own properties, and an embedded message attachment nests a whole message structure inside the parent. Beyond the fixed MAPI tag range, a PST carries a named-property map that assigns numeric IDs to string- or GUID-named properties, which is how custom and third-party fields survive in the file. Export a single message out of this graph and you get a MSG file; the offline cache of an Exchange or Microsoft 365 mailbox uses the identical layered format in an OST file.

Data encoding and the weak password scheme

The header byte bCryptMethod declares how the data inside blocks is encoded. NDB_CRYPT_NONE (0x00) leaves blocks in the clear. NDB_CRYPT_PERMUTE (0x01) runs each data byte through a fixed 256-entry substitution table. NDB_CRYPT_CYCLIC (0x02) adds a position- and key-dependent rotation on top. Both are byte-level obfuscation with a fixed, published algorithm and no secret key. They deter casual inspection with a hex editor; they are not cryptography, and any tool that implements [MS-PST] decodes them without a password.

PST “password protection” is weaker still, and this matters because a PST holds an entire mailbox of sensitive personal and business data. Outlook does not store the password or use it to derive an encryption key. It stores only a CRC-32 hash of the password string in the PidTagPstPassword property (tag 0x67FF0003) in the message store. On open, Outlook compares the CRC-32 of what you type against that stored value; the actual message data is not keyed to the password at all. Because CRC-32 is a 32-bit checksum with abundant collisions, many different strings hash to the same value, so more than one password unlocks the same file, and freely available tools recover a working password or strip the field entirely in seconds. Treat a PST as confidential by controlling the file itself (disk encryption, access control), not by relying on the built-in password.

Integrity checks and how a PST becomes unreadable

A PST defends itself with several layers of checksums. The header carries dwCRCPartial and, in Unicode files, dwCRCFull. Every data block ends with a trailer that stores the block’s length, a signature derived from its BID and offset, and a CRC over the block contents; every b-tree page has a matching trailer. When Outlook reads a block whose stored CRC or signature does not match, it treats the file as damaged.

Corruption usually comes from a truncated write, a full disk, an interrupted sync, or an ANSI PST that reached its 2 GB ceiling. Microsoft ships ScanPST.exe, the Inbox Repair Tool, with Outlook for exactly this case: it walks the NBT and BBT, discards nodes and blocks whose checksums fail, rebuilds the tree references, and writes a repaired file (after backing up the original). It is the first thing to try when a PST will not open. An orphaned OST cannot be repaired the same way and reattached to a new machine, because an OST is bound to its Exchange account; recovering its mail means converting it to a PST.

Frequently asked questions

How can I tell an ANSI PST from a Unicode PST?

Read the 2-byte wVer field at offset 0x000A. A value of 14 or 15 is an ANSI PST, capped at 2 GB; a value of 23 or higher is a Unicode PST with 64-bit offsets and a much larger ceiling. Both begin with the same 21 42 44 4E (!BDN) signature at offset 0, so the signature alone does not tell them apart.

Why can a PST move between computers but an OST cannot?

A PST is a self-contained, account-independent store: the file holds the complete mailbox, so copying it to another machine and opening it in Outlook just works. An OST uses the same on-disk format but is the local cache of one specific Exchange or Microsoft 365 account; it is keyed to that account and Outlook profile, so it cannot simply be mounted elsewhere and must be reconnected to its account or converted to a PST.

Why is the PST password so easy to bypass?

Because it never encrypts anything. Outlook stores only a CRC-32 hash of the password in the PidTagPstPassword property and checks it on open; the message data itself is not encrypted with a key derived from the password. CRC-32 collisions mean several passwords match the same hash, so recovery tools return a working password or delete the property outright.

References