<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Khola.Blog]]></title><description><![CDATA[Systems & Abstractions: Deep-dive architectural reviews of systems and AI texts.]]></description><link>https://www.khola.blog</link><image><url>https://substackcdn.com/image/fetch/$s_!xafK!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F39fd7ea9-21c3-4547-a047-5c3ec1c25c8f_1280x1280.png</url><title>Khola.Blog</title><link>https://www.khola.blog</link></image><generator>Substack</generator><lastBuildDate>Fri, 15 May 2026 09:39:56 GMT</lastBuildDate><atom:link href="https://www.khola.blog/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Nitin Khola]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[nitinkhola@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[nitinkhola@substack.com]]></itunes:email><itunes:name><![CDATA[Nitin Khola]]></itunes:name></itunes:owner><itunes:author><![CDATA[Nitin Khola]]></itunes:author><googleplay:owner><![CDATA[nitinkhola@substack.com]]></googleplay:owner><googleplay:email><![CDATA[nitinkhola@substack.com]]></googleplay:email><googleplay:author><![CDATA[Nitin Khola]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[Designing Data-Intensive Applications in 2026: An Architectural Retrospective
]]></title><description><![CDATA[A definitive review of Martin Kleppmann's foundational text, updating its distributed systems theory for the modern era of NVMe storage, unified lakehouses, and orchestrated sagas.]]></description><link>https://www.khola.blog/p/designing-data-intensive-applications</link><guid isPermaLink="false">https://www.khola.blog/p/designing-data-intensive-applications</guid><dc:creator><![CDATA[Nitin Khola]]></dc:creator><pubDate>Thu, 14 May 2026 03:18:17 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!mh-h!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!mh-h!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!mh-h!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic 424w, https://substackcdn.com/image/fetch/$s_!mh-h!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic 848w, https://substackcdn.com/image/fetch/$s_!mh-h!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic 1272w, https://substackcdn.com/image/fetch/$s_!mh-h!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!mh-h!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic" width="304" height="398.848" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:656,&quot;width&quot;:500,&quot;resizeWidth&quot;:304,&quot;bytes&quot;:62187,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/heic&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.khola.blog/i/197620339?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!mh-h!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic 424w, https://substackcdn.com/image/fetch/$s_!mh-h!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic 848w, https://substackcdn.com/image/fetch/$s_!mh-h!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic 1272w, https://substackcdn.com/image/fetch/$s_!mh-h!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9aff54a1-8d8d-4acb-9a35-fe5903d22b17_500x656.heic 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Martin Kleppmann&#8217;s <em>Designing Data-Intensive Applications</em> is basically required reading for backend engineering. The core premise still holds up: data volume, complexity, and velocity constrain architecture far more than raw CPU cycles.</p><p>But hardware doesn&#8217;t stand still. Reading the book in 2026 requires updating the physical variables Kleppmann used in his assumptions. When you swap in modern NVMe storage and unified compute engines, the structural trade-offs he outlines shift significantly. Here is a look at what actually survives contact with modern production infrastructure.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.khola.blog/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Khola.Blog! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h3><strong>1. Storage: The Metal Layer</strong></h3><p>Kleppmann dedicates a lot of space to the trade-offs between the traditional B-Tree and the Log-Structured Merge-tree (LSM-tree).</p><p>The B-Tree updates fixed-size logical disk pages in-place. DDIA correctly points out why this was a problem on magnetic hard disk drives (HDDs). Modifying a small record meant reading an entire page into memory, updating it, and writing it back to random locations. Because HDDs rely on mechanical arms, random seeks killed your IOPS.</p><p>LSM-trees bypassed this by turning random writes into sequential logs. Data is appended to a Write-Ahead Log and a memory buffer, then flushed to disk as immutable Sorted String Tables. It solved the HDD write bottleneck but introduced the compaction tax. Background compaction burns CPU and saturates disk I/O, which causes unpredictable tail latency spikes.</p><p><strong>The 2026 Reality:</strong> The modern data center runs on enterprise NVMe SSDs. These drives use highly parallel flash channels that largely neutralize the performance gap between sequential and random writes.</p><p>More importantly, modern enterprise SSDs have internal hardware controllers that perform transparent compression directly on the physical I/O path. If you use a modern B-Tree variant that logs zero-padded delta updates, the NVMe hardware instantly compresses the zeroes away. This drops the write amplification of the B-Tree from over 200x down to roughly 20x.</p><p>For general-purpose workloads today, the B-Tree is usually the better default. You get predictable read performance without paying the LSM compaction tax, because the hardware solved the random-write problem for you.</p><h3><strong>2. Distributed Data: The Network Layer</strong></h3><p>When dealing with distributed transactions, DDIA covers coordination protocols like Two-Phase Commit (2PC). In 2PC, a coordinator asks all participating nodes to place pessimistic locks on database resources before committing.</p><p>While the book acknowledges the performance hit of 2PC, its blocking nature is a complete non-starter in modern async architectures.</p><p><strong>The 2026 Reality:</strong> Today&#8217;s infrastructure relies heavily on long-running AI workflows and agent swarms. A single workflow might take hours to resolve. You cannot hold synchronous database locks across a distributed cluster for hours without causing complete system gridlock.</p><p>The industry has largely abandoned strong global consistency in favor of eventual consistency. We rely on Orchestrated Sagas and the transactional outbox pattern. You isolate the transaction locally, commit it, and publish an event to trigger the next step. There are no distributed locks held across service boundaries.</p><h3><strong>3. Derived Data: The Unbundling Layer</strong></h3><p>One of the best concepts in DDIA is unbundling the database. Instead of treating a single RDBMS as the system of record and fighting to keep caches in sync, you treat an append-only event log (like Kafka) as the source of truth. The database and search indexes are just materialized views derived from that log.</p><p>To handle analytical workloads in this model, DDIA discusses the Lambda Architecture. This involves maintaining a slow batch pipeline for accuracy and a fast stream pipeline for real-time approximation.</p><p><strong>The 2026 Reality:</strong> In practice, Lambda was an operational nightmare. You had to write, debug, and maintain your business logic twice across two different frameworks. That pattern is dead.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!9hdP!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!9hdP!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png 424w, https://substackcdn.com/image/fetch/$s_!9hdP!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png 848w, https://substackcdn.com/image/fetch/$s_!9hdP!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png 1272w, https://substackcdn.com/image/fetch/$s_!9hdP!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!9hdP!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png" width="1456" height="664" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:664,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:348555,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.khola.blog/i/197620339?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!9hdP!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png 424w, https://substackcdn.com/image/fetch/$s_!9hdP!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png 848w, https://substackcdn.com/image/fetch/$s_!9hdP!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png 1272w, https://substackcdn.com/image/fetch/$s_!9hdP!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9412d128-f382-4939-bc65-f33750dc13cd_3316x1512.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The maturation of unified execution models like Apache Flink operating over Lakehouse formats like Apache Iceberg has eliminated the need for it. &#8220;Batch&#8221; is now just treated as a bounded stream operating over a finite dataset. You get ACID transaction guarantees directly on top of scalable object storage. You write the pipeline once, and it handles both real-time ingestion and historical batch queries using snapshot isolation.</p><h3><strong>Final Thoughts</strong></h3><p>DDIA is still the best baseline we have for distributed systems theory. The abstractions are correct. But as engineers, we can&#8217;t treat the implementation details as dogma. The physical hardware dictates the architecture. Update the hardware assumptions, and the correct architecture changes with it.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.khola.blog/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Khola.Blog! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item></channel></rss>