skip to content
ksegit
LON --:--:--

BigIntCsharp

A zero-dependency C# parser for Apache Parquet INT96 timestamps - 12 bytes of deprecated-but-everywhere format into .NET DateTime, with endianness handled correctly.

c# · .net · parquet · bignumber

Problem

Parquet's INT96 timestamp - 8 bytes of nanoseconds-of-day plus 4 bytes of Julian day - was deprecated years ago and still shows up constantly in real data lakes (Spark and Impala wrote it for years). .NET tooling for it is thin, the byte order is little-endian in a format people read as hex strings, and getting it wrong shifts timestamps by whole eras rather than milliseconds.

Approach

The itch came from real ETL work on large Parquet datasets - pipelines that partition and validate big-data workflows can't afford a mis-parsed timestamp column.

A single-file, zero-dependency library (System.Numerics only) exposing three paths: hex-string parsing, BigInteger parsing, and a static quick-conversion helper.

 0x8096980000000000E8030000            (INT96, 12 bytes)
 └───────┬───────┘└───┬───┘
   nanoseconds of day  Julian day      (little-endian!)
         │                │
   byte reverse     byte reverse
         │                │
   ns → TimeSpan    JD → Gregorian date
         └────────┬───────┘
              .NET DateTime (ns precision)

Julian-day ranges and input format are validated up front - malformed input fails loudly instead of producing a plausible wrong date.

Tradeoffs

  • Copy-the-file distribution. One .cs file instead of a NuGet package: zero dependency-chain surface, at the cost of manual updates.
  • BigInteger convenience. The BigInteger path reads naturally next to data-engineering code; raw byte slicing would be faster but this parser lives at ETL edges, not in hot loops.

Outcome

MIT-licensed, documented with worked examples, and doing its job wherever INT96 refuses to die.