• About TC
  • Affiliate Disclaimer
  • Privacy Policy
  • TOS
  • Contact
Tuesday, July 8, 2025
Techcratic
  • TC
  • AI
    Artificial Intelligence

    Build a just-in-time knowledge base with Amazon Bedrock

    Artificial Intelligence

    Transforming network operations with AI: How Swisscom built a network assistant using Amazon Bedrock

    Artificial Intelligence

    EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video

    Artificial Intelligence

    Instruction-Following Pruning for Large Language Models

    Artificial Intelligence

    How to Combine Streamlit, Pandas, and Plotly for Interactive Data Apps

    Artificial Intelligence

    Tailor responsible AI with new safeguard tiers in Amazon Bedrock Guardrails

    Artificial Intelligence

    Automate Data Quality Reports with n8n: From CSV to Professional Analysis

    Artificial Intelligence

    NewDay builds A Generative AI based Customer service Agent Assist with over 90% accuracy

    Artificial Intelligence

    5 Things You Need to Know About Agentic AI

  • Apple
    iPhone 17 Air rumor: New sky blue color

    iPhone 17 Air rumor: New sky blue color

    Apple @ Work Podcast: Fleet launches iOS device management

    Apple @ Work Podcast: Portability for Passkeys is coming to macOS

    macOS Tahoe 26 beta 3 now available, here’s what to expect

    macOS Tahoe 26 beta 3 now available, here’s what to expect

    Apple releases developer beta 3 for visionOS 26

    Apple releases developer beta 3 for visionOS 26

    Apple releases watchOS 26 beta 3, here’s what to expect

    Apple releases watchOS 26 beta 3, here’s what to expect

    Apple’s AI agent can describe Street View scenes to blind people

    Apple’s AI agent can describe Street View scenes to blind people

    Score AirPods from just $89

    Score AirPods from just $89

    Following hiccup, macOS 26 beta 3 now available for Apple Silicon

    Following hiccup, macOS 26 beta 3 now available for Apple Silicon

    Kingston IronKey Vault Privacy 80 review: Specs, features, price

    Kingston IronKey Vault Privacy 80 review: Specs, features, price

  • Gaming
    Police Checkpoint |As Dusk Falls Ep.25

    Police Checkpoint |As Dusk Falls Ep.25

    How to get the Amber Mutation in Grow a Garden

    How to get the Amber Mutation in Grow a Garden

    Diablo 4 Beta Review – Should It Be Delayed?

    Diablo 4 Beta Review – Should It Be Delayed?

    Turn your wall into the ultimate PC gaming display with these Prime Day Valerion projector deals

    Turn your wall into the ultimate PC gaming display with these Prime Day Valerion projector deals

    Sonic Frontiers: Was Update 2 ACTUALLY Good? Review

    Let's Talk About Gotham Knights’ Predictable Ending

    Let's Talk About Gotham Knights’ Predictable Ending

    Linus Torvalds has apparently met Bill Gates for the first time in person and before you ask, no he didn’t clock him in the face

    Racks packing Nvidia’s newest and shiniest AI supercomputer Blackwell Ultra cards have just been deployed by CoreWeave

    Gungrave G.O.R.E | Gameplay Walkthrough Part 32  No commentary (PC)

    Gungrave G.O.R.E | Gameplay Walkthrough Part 32 No commentary (PC)

    Linus Torvalds has apparently met Bill Gates for the first time in person and before you ask, no he didn’t clock him in the face

    This striking two-toned mini PC features a fully customised fanless cooling system for Framework and AMD’s new Halo Strix motherboards

  • Retro Rewind
    Retro Rewind: Electronic Games April 1995

    Retro Rewind: Electronic Games April 1995

    Retro Rewind: Electronic Gaming Monthly Magazine Number 55 February 1994

    Retro Rewind: Electronic Gaming Monthly Magazine Number 57 April 1994

    Retro Rewind: Blast from the Past – 35 Iconic Commercials of 1988!

    Retro Rewind: Blast from the Past – 35 Iconic Commercials of 1988!

    Retro Rewind: PC World Magazine August 1998

    Retro Rewind: PC World Magazine August 1998

    Retro Rewind: Computer Shopper Magazine September 1997

    Retro Rewind: Computer Shopper Magazine September 1997

    Retro Rewind: PC Magazine December 2015

    Retro Rewind: PC Magazine December 2015

    Retro Rewind: EDGE Magazine RETRO #1: The Guide to Classic Videogame Playing and Collecting

    Retro Rewind: EDGE Magazine RETRO #1: The Guide to Classic Videogame Playing and Collecting

    Retro Rewind: Computer Gaming World Magazine Issue 73 December 1998

    Retro Rewind: Computer Gaming World Magazine Issue 73 December 1998

    Retro Rewind: Electronic Gaming Monthly Magazine Number 55 February 1994

    Retro Rewind: Electronic Gaming Monthly Magazine Number 55 February 1994

  • Tech Deals
    Long HDMI 2.1 Cable 30FT – 8K 60hz,Ultra High Speed HDMI Cable 4K 120hz…

    Long HDMI 2.1 Cable 30FT – 8K 60hz,Ultra High Speed HDMI Cable 4K 120hz…

    acer 7-in-1 Wireless Charging Station: Charger Stand for Multiple Devices for Apple -…

    acer 7-in-1 Wireless Charging Station: Charger Stand for Multiple Devices for Apple -…

    Rii RK100+ Multiple Color Rainbow LED Backlit Large Size USB Wired Mechanical Feeling…

    Rii RK100+ Multiple Color Rainbow LED Backlit Large Size USB Wired Mechanical Feeling…

    FC100 Gaming Headset with Microphone for PS4, PS5, Switch, Xbox, PC, Mac, Gamer Headset…

    FC100 Gaming Headset with Microphone for PS4, PS5, Switch, Xbox, PC, Mac, Gamer Headset…

    Killzone 2 – Playstation 3 (Renewed)

    Killzone 2 – Playstation 3 (Renewed)

    StarTech.com 10ft CAT6a Ethernet Cable – 10 Gigabit Shielded Snagless RJ45 100W PoE…

    StarTech.com 10ft CAT6a Ethernet Cable – 10 Gigabit Shielded Snagless RJ45 100W PoE…

    TAGRY Bluetooth Headphones True Wireless Earbuds 60H Playback LED Power Display…

    TAGRY Bluetooth Headphones True Wireless Earbuds 60H Playback LED Power Display…

    Samsung SSD 9100 PRO 4TB, PCIe 5.0×4 M.2 2280, Seq. Read Speeds Up to 14,800MB/s, Best…

    Samsung SSD 9100 PRO 4TB, PCIe 5.0×4 M.2 2280, Seq. Read Speeds Up to 14,800MB/s, Best…

    ORICO 2TB NVMe SSD PCIe 4.0 – Up to 7000MB/s, M.2 2280 Internal Solid State Drive, Fast…

    ORICO 2TB NVMe SSD PCIe 4.0 – Up to 7000MB/s, M.2 2280 Internal Solid State Drive, Fast…

  • Tesla
    Side View Mirror Cover with Tesla Cybertruck,Matte Carbon Fiber Pattern Rearview Mirror…

    Side View Mirror Cover with Tesla Cybertruck,Matte Carbon Fiber Pattern Rearview Mirror…

    For Tesla Model Y 2025 2026 Juniper& 2024-2025 Model 3 Highland Glossy Screen Protector…

    For Tesla Model Y 2025 2026 Juniper& 2024-2025 Model 3 Highland Glossy Screen Protector…

    Uireefly Center Console Wireless Charging Mat for Tesla Cybertruck 2024, Anti-Slip…

    Uireefly Center Console Wireless Charging Mat for Tesla Cybertruck 2024, Anti-Slip…

    Car Tow Hook Decor, V Shape Trailer Hook Sticker for Car Bumper, Auto Bumper Trailer Tow…

    Car Tow Hook Decor, V Shape Trailer Hook Sticker for Car Bumper, Auto Bumper Trailer Tow…

    Tesla employees try to oust Elon, new Volvo, and Micah’s close call

    America – it’s a party now! Plus: an electric Honda Ruckus!

    Heavy Duty D-Rings For Tesla Cybertruck 2023 2024 2025 4pcs Ring Truck Bed Hooks…

    Heavy Duty D-Rings For Tesla Cybertruck 2023 2024 2025 4pcs Ring Truck Bed Hooks…

    Tesla was forced to reimburse Full Self-Driving in arbitration after failing to deliver

    3D MAXpider Custom Fit Kagu Floor Mat (Black) Compatible with Tesla Model S 2021-2025 -…

    3D MAXpider Custom Fit Kagu Floor Mat (Black) Compatible with Tesla Model S 2021-2025 -…

    Tesla will pause part of new Model Y production for 3 weeks for upgrades, report says

    Elon Musk realized too late how badly Tesla will hurt from Trump’s bill

  • UFO
    Sun Halo A Natural Phenomena #viralvideo

    Sun Halo A Natural Phenomena #viralvideo

    BulbaCraft UFO Stickers, Alien Stickers for Water Bottle and Laptop – UFO Party Favors & Decorations, Waterproof Vinyl Decals, Gifts for Women & Men

    BulbaCraft UFO Stickers, Alien Stickers for Water Bottle and Laptop – UFO Party Favors & Decorations, Waterproof Vinyl Decals, Gifts for Women & Men

    I Married a Monster From Outer Space

    I Married a Monster From Outer Space

    Space Theme Birthday Candle, Shiny Astronaut Number Candle Spaceship Outer Space Cake Topper Perfect Universe Rocket Spacecraft Cake Decorations and Party Favors(Number 6)

    Space Theme Birthday Candle, Shiny Astronaut Number Candle Spaceship Outer Space Cake Topper Perfect Universe Rocket Spacecraft Cake Decorations and Party Favors(Number 6)

    Paranormal Phenomena: Modern Tales of the Unexplained

    Paranormal Phenomena: Modern Tales of the Unexplained

    Top 5 UFO Encounters of 2025 | New Evidence Emerges

    Top 5 UFO Encounters of 2025 | New Evidence Emerges

    Electric Disc Launcher with 6 Flying Spinners, Glow in The Dark Rechargeable Outdoor Play Fun Sport Fidget Toys

    Electric Disc Launcher with 6 Flying Spinners, Glow in The Dark Rechargeable Outdoor Play Fun Sport Fidget Toys

    UFO COMMUNITY IN A FRENZY – “Like Something Out Of An Alien Movie” | Proof Is Out There | #Shorts

    Evil Brain From Outer Space (Retro Cover Art) [DVD]

    Evil Brain From Outer Space (Retro Cover Art) [DVD]

No Result
View All Result
  • TC
  • AI
    Artificial Intelligence

    Build a just-in-time knowledge base with Amazon Bedrock

    Artificial Intelligence

    Transforming network operations with AI: How Swisscom built a network assistant using Amazon Bedrock

    Artificial Intelligence

    EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video

    Artificial Intelligence

    Instruction-Following Pruning for Large Language Models

    Artificial Intelligence

    How to Combine Streamlit, Pandas, and Plotly for Interactive Data Apps

    Artificial Intelligence

    Tailor responsible AI with new safeguard tiers in Amazon Bedrock Guardrails

    Artificial Intelligence

    Automate Data Quality Reports with n8n: From CSV to Professional Analysis

    Artificial Intelligence

    NewDay builds A Generative AI based Customer service Agent Assist with over 90% accuracy

    Artificial Intelligence

    5 Things You Need to Know About Agentic AI

  • Apple
    iPhone 17 Air rumor: New sky blue color

    iPhone 17 Air rumor: New sky blue color

    Apple @ Work Podcast: Fleet launches iOS device management

    Apple @ Work Podcast: Portability for Passkeys is coming to macOS

    macOS Tahoe 26 beta 3 now available, here’s what to expect

    macOS Tahoe 26 beta 3 now available, here’s what to expect

    Apple releases developer beta 3 for visionOS 26

    Apple releases developer beta 3 for visionOS 26

    Apple releases watchOS 26 beta 3, here’s what to expect

    Apple releases watchOS 26 beta 3, here’s what to expect

    Apple’s AI agent can describe Street View scenes to blind people

    Apple’s AI agent can describe Street View scenes to blind people

    Score AirPods from just $89

    Score AirPods from just $89

    Following hiccup, macOS 26 beta 3 now available for Apple Silicon

    Following hiccup, macOS 26 beta 3 now available for Apple Silicon

    Kingston IronKey Vault Privacy 80 review: Specs, features, price

    Kingston IronKey Vault Privacy 80 review: Specs, features, price

  • Gaming
    Police Checkpoint |As Dusk Falls Ep.25

    Police Checkpoint |As Dusk Falls Ep.25

    How to get the Amber Mutation in Grow a Garden

    How to get the Amber Mutation in Grow a Garden

    Diablo 4 Beta Review – Should It Be Delayed?

    Diablo 4 Beta Review – Should It Be Delayed?

    Turn your wall into the ultimate PC gaming display with these Prime Day Valerion projector deals

    Turn your wall into the ultimate PC gaming display with these Prime Day Valerion projector deals

    Sonic Frontiers: Was Update 2 ACTUALLY Good? Review

    Let's Talk About Gotham Knights’ Predictable Ending

    Let's Talk About Gotham Knights’ Predictable Ending

    Linus Torvalds has apparently met Bill Gates for the first time in person and before you ask, no he didn’t clock him in the face

    Racks packing Nvidia’s newest and shiniest AI supercomputer Blackwell Ultra cards have just been deployed by CoreWeave

    Gungrave G.O.R.E | Gameplay Walkthrough Part 32  No commentary (PC)

    Gungrave G.O.R.E | Gameplay Walkthrough Part 32 No commentary (PC)

    Linus Torvalds has apparently met Bill Gates for the first time in person and before you ask, no he didn’t clock him in the face

    This striking two-toned mini PC features a fully customised fanless cooling system for Framework and AMD’s new Halo Strix motherboards

  • Retro Rewind
    Retro Rewind: Electronic Games April 1995

    Retro Rewind: Electronic Games April 1995

    Retro Rewind: Electronic Gaming Monthly Magazine Number 55 February 1994

    Retro Rewind: Electronic Gaming Monthly Magazine Number 57 April 1994

    Retro Rewind: Blast from the Past – 35 Iconic Commercials of 1988!

    Retro Rewind: Blast from the Past – 35 Iconic Commercials of 1988!

    Retro Rewind: PC World Magazine August 1998

    Retro Rewind: PC World Magazine August 1998

    Retro Rewind: Computer Shopper Magazine September 1997

    Retro Rewind: Computer Shopper Magazine September 1997

    Retro Rewind: PC Magazine December 2015

    Retro Rewind: PC Magazine December 2015

    Retro Rewind: EDGE Magazine RETRO #1: The Guide to Classic Videogame Playing and Collecting

    Retro Rewind: EDGE Magazine RETRO #1: The Guide to Classic Videogame Playing and Collecting

    Retro Rewind: Computer Gaming World Magazine Issue 73 December 1998

    Retro Rewind: Computer Gaming World Magazine Issue 73 December 1998

    Retro Rewind: Electronic Gaming Monthly Magazine Number 55 February 1994

    Retro Rewind: Electronic Gaming Monthly Magazine Number 55 February 1994

  • Tech Deals
    Long HDMI 2.1 Cable 30FT – 8K 60hz,Ultra High Speed HDMI Cable 4K 120hz…

    Long HDMI 2.1 Cable 30FT – 8K 60hz,Ultra High Speed HDMI Cable 4K 120hz…

    acer 7-in-1 Wireless Charging Station: Charger Stand for Multiple Devices for Apple -…

    acer 7-in-1 Wireless Charging Station: Charger Stand for Multiple Devices for Apple -…

    Rii RK100+ Multiple Color Rainbow LED Backlit Large Size USB Wired Mechanical Feeling…

    Rii RK100+ Multiple Color Rainbow LED Backlit Large Size USB Wired Mechanical Feeling…

    FC100 Gaming Headset with Microphone for PS4, PS5, Switch, Xbox, PC, Mac, Gamer Headset…

    FC100 Gaming Headset with Microphone for PS4, PS5, Switch, Xbox, PC, Mac, Gamer Headset…

    Killzone 2 – Playstation 3 (Renewed)

    Killzone 2 – Playstation 3 (Renewed)

    StarTech.com 10ft CAT6a Ethernet Cable – 10 Gigabit Shielded Snagless RJ45 100W PoE…

    StarTech.com 10ft CAT6a Ethernet Cable – 10 Gigabit Shielded Snagless RJ45 100W PoE…

    TAGRY Bluetooth Headphones True Wireless Earbuds 60H Playback LED Power Display…

    TAGRY Bluetooth Headphones True Wireless Earbuds 60H Playback LED Power Display…

    Samsung SSD 9100 PRO 4TB, PCIe 5.0×4 M.2 2280, Seq. Read Speeds Up to 14,800MB/s, Best…

    Samsung SSD 9100 PRO 4TB, PCIe 5.0×4 M.2 2280, Seq. Read Speeds Up to 14,800MB/s, Best…

    ORICO 2TB NVMe SSD PCIe 4.0 – Up to 7000MB/s, M.2 2280 Internal Solid State Drive, Fast…

    ORICO 2TB NVMe SSD PCIe 4.0 – Up to 7000MB/s, M.2 2280 Internal Solid State Drive, Fast…

  • Tesla
    Side View Mirror Cover with Tesla Cybertruck,Matte Carbon Fiber Pattern Rearview Mirror…

    Side View Mirror Cover with Tesla Cybertruck,Matte Carbon Fiber Pattern Rearview Mirror…

    For Tesla Model Y 2025 2026 Juniper& 2024-2025 Model 3 Highland Glossy Screen Protector…

    For Tesla Model Y 2025 2026 Juniper& 2024-2025 Model 3 Highland Glossy Screen Protector…

    Uireefly Center Console Wireless Charging Mat for Tesla Cybertruck 2024, Anti-Slip…

    Uireefly Center Console Wireless Charging Mat for Tesla Cybertruck 2024, Anti-Slip…

    Car Tow Hook Decor, V Shape Trailer Hook Sticker for Car Bumper, Auto Bumper Trailer Tow…

    Car Tow Hook Decor, V Shape Trailer Hook Sticker for Car Bumper, Auto Bumper Trailer Tow…

    Tesla employees try to oust Elon, new Volvo, and Micah’s close call

    America – it’s a party now! Plus: an electric Honda Ruckus!

    Heavy Duty D-Rings For Tesla Cybertruck 2023 2024 2025 4pcs Ring Truck Bed Hooks…

    Heavy Duty D-Rings For Tesla Cybertruck 2023 2024 2025 4pcs Ring Truck Bed Hooks…

    Tesla was forced to reimburse Full Self-Driving in arbitration after failing to deliver

    3D MAXpider Custom Fit Kagu Floor Mat (Black) Compatible with Tesla Model S 2021-2025 -…

    3D MAXpider Custom Fit Kagu Floor Mat (Black) Compatible with Tesla Model S 2021-2025 -…

    Tesla will pause part of new Model Y production for 3 weeks for upgrades, report says

    Elon Musk realized too late how badly Tesla will hurt from Trump’s bill

  • UFO
    Sun Halo A Natural Phenomena #viralvideo

    Sun Halo A Natural Phenomena #viralvideo

    BulbaCraft UFO Stickers, Alien Stickers for Water Bottle and Laptop – UFO Party Favors & Decorations, Waterproof Vinyl Decals, Gifts for Women & Men

    BulbaCraft UFO Stickers, Alien Stickers for Water Bottle and Laptop – UFO Party Favors & Decorations, Waterproof Vinyl Decals, Gifts for Women & Men

    I Married a Monster From Outer Space

    I Married a Monster From Outer Space

    Space Theme Birthday Candle, Shiny Astronaut Number Candle Spaceship Outer Space Cake Topper Perfect Universe Rocket Spacecraft Cake Decorations and Party Favors(Number 6)

    Space Theme Birthday Candle, Shiny Astronaut Number Candle Spaceship Outer Space Cake Topper Perfect Universe Rocket Spacecraft Cake Decorations and Party Favors(Number 6)

    Paranormal Phenomena: Modern Tales of the Unexplained

    Paranormal Phenomena: Modern Tales of the Unexplained

    Top 5 UFO Encounters of 2025 | New Evidence Emerges

    Top 5 UFO Encounters of 2025 | New Evidence Emerges

    Electric Disc Launcher with 6 Flying Spinners, Glow in The Dark Rechargeable Outdoor Play Fun Sport Fidget Toys

    Electric Disc Launcher with 6 Flying Spinners, Glow in The Dark Rechargeable Outdoor Play Fun Sport Fidget Toys

    UFO COMMUNITY IN A FRENZY – “Like Something Out Of An Alien Movie” | Proof Is Out There | #Shorts

    Evil Brain From Outer Space (Retro Cover Art) [DVD]

    Evil Brain From Outer Space (Retro Cover Art) [DVD]

No Result
View All Result
Techcratic
No Result
View All Result
Home Hacker News

It’s Time to Stop Building KV Databases • Buttondown

Hacker News by Hacker News
March 25, 2025
in Hacker News
Reading Time: 11 mins read
129
A A
0

2025-03-24 16:31:00
buttondown.com

March 24, 2025

NULL BITMAP.png

I’m so sick of Key-Value. The most unopinionated, uninspired data model that is a pain to use for anything.

They’re very popular! Among database vendors. Or, specifically, storage engine vendors. They love to offer you a byte array to byte array map, and if they are feeling particularly generous, it might even be ordered so that you can do range scans.

The problem is that the only thing that a KV interface is good for is as a building block for a reasonable data model. This being the primitive means that everyone who uses a KV database has to do that from scratch. And they (read: me) usually do a middling job of it, because it’s not actually the problem they were setting out to solve. Yes: we could have a third-party library that is built to sit atop RocksDB. That is the natural “separation of concerns” answer. But look: are you going to use some guy’s Rust library with 300 dependencies in its Cargo.toml and an anime logo? Because I’m not. I want to use a singular database to the extent possible and I don’t want that database to foist the responsibility of more dependencies onto me.

“Data independence” is, to me, the core selling point of the relational model. Data independence comprises rules 8 and 9 of Codd’s twelve rules for relational databases. From Wikipedia:

Rule 8: Physical data independence:

Application programs and terminal activities remain logically unimpaired whenever any changes are made in either storage representations or access methods.

Rule 9: Logical data independence:

Application programs and terminal activities remain logically unimpaired when information-preserving changes of any kind that theoretically permit unimpairment are made to the base tables.

This draws a distinction between the physical schema, which constitutes things like data encoding and index structure, and logical schema, which represents things like which columns are in a given table and what their names and types and constraints are. The platonic relational view of the world is that queries should always be written against the logical schema, and never against the physical.

The piece of a database that enables data independence is the query planner. The thing that takes a declarative statement of what the user would like to achieve and translates it into an imperative plan. It’s this translation layer that protects application code from knowing details about how data is stored.

I have heard repeatedly from many users of databases that they like KV over SQL because they don’t like to wade through a query planner in order to get their queries to work. This is absolutely justified! For a lot of use cases it’s simply not appropriate to declaratively specify your computation. You need to know that you are doing a constrained range scan, not that “if I write the SQL this way, the database is very likely to execute it as a constrained range scan.” My contention is that this is a false dilemma: we can achieve something better than “byte to byte array” without going all the way up to “relational utopia data independence.”

My preferred base data model would be records. Typed records that you might find in a relational database consisting of both a logical level describing “here are the columns I want access to” and a physical level describing “here are sets of columns I want to be able to query by.” That is, a decoupling of the logical schema and physical schema.

More simply:

  • KV: no distinction between logical and physical schema.
  • Relational Model: logical and physical schema are distinct, queries are written again the logical schema.
  • My proposed sweet spot for embedded databases: logical and physical schema are distinct, queries are written against the physical schema.

Let’s get more concrete. We might create a table in SQL like this:

-- Logical schema
CREATE TABLE data (
  ts TIMESTAMP NOT NULL,
  id INT NOT NULL,
  user_id INT NOT NULL,
  reason TEXT NOT NULL
);

-- Physical schema
CREATE UNIQUE INDEX ON data primary_idx (ts, id);

This is fine, and normal, and maps cleanly onto a KV database like RocksDB: you just make the key some kind of encoding of (id, ts) which preserves lexicographic comparisons (which is easy for fixed-size types, and a little tricky for variable-sized types), and then you make the value some concatenation of user_id and reason.

The concern with actually using a SQL database that can arise if you have other indexes:

CREATE INDEX ON data secondary_user_id_idx (user_id, ts);
CREATE INDEX ON data secondary_reason_idx (reason, ts);

And I write a SQL query like:

SELECT * FROM data WHERE ts > 100 AND ts  200 AND user_id = 4 AND reason = 'expired';

There are two natural plans for this query:

  1. Scan the index secondary_user_id_idx restricting user_id and ts, having a residual filter on the reason column or
  2. scan the index secondary_reason_idx restricting reason and ts, having a residual filter on the user_id column.

Picking between these comes down to knowing if reason_id or user_id are more selective. We want to pick the option that will force us to process fewer rows with our residual filter on top. This is the job of the query planner.

The objection to query planners in general comes down to not wanting this “smart” layer in between the authoring of a query and its execution. There is a risk here that the query planner will make the wrong choice and we won’t have an easy way to predict or correct that. For a very limited use case, where we know for the most part, ahead of time what our access patterns will be, and have a small set of queries, this risk (and headache) might not be worth the benefits of data independence. This is why people sometimes argue in favour of KV databases, it means this kind of thing doesn’t come up.

We don’t have to go all the way to that, though. We can have a lot of the nice benefits of records and avoid these headaches by simply…not having a query planner. If you don’t mind writing query plans explicitly (and writing code to process KV data is strictly easier than writing query plans), then this is completely fine. You just make your query language require you to specify what index you want to read and in what order you want operations performed.

You don’t get the benefits of a real query planner in that you don’t get to dynamically respond to data distribution changes, and you don’t get to specialize queries based on their parameters, and you need to at once understand the physical and logical schema of your database, but these are all things that are still true of a KV database.

If your database understands your schema, here are some of the things it is now able to do for you:

  • Physical schema changes such as secondary indexing, including asynchronous secondary index builds.
  • Logical schema changes, like adding or deleting new columns.
  • Transparent swapping between row- and column-oriented layouts without requiring code changes. Perhaps even co-existing within the same database.
  • Save you from implementing the ad hoc, informally-specified, bug-ridden, slow implementation of column encoding you were probably going to write anyway.

Of course, there are good reasons to provide the most unopinionated interface possible. If you want to provide, say, range scans, well, now you are shipping a type system and semantics on your data. Every decision you have to make is an opportunity to make the wrong one.

Perhaps the biggest decision that has to be made is “how deeply do your records permeate your data model,” meaning, do you have types that you write to disk and then know how to compare? Or do you have types that you know how to convert into byteslices that will then compare lexicographically. Both options come with tradeoffs. I’m not aware of a perfect solution here.

Despite that, I think there is a place for embedded databases that has a more opinionated view on how data should be stored. SQLite is sort of this! SQLite doesn’t get floated as a RocksDB alternative all that often but it actually is that, in a lot of ways. I think it is perhaps hampered by the fact that it is SQL and so has a lot of bad connotations for the kind of workloads people would want to use RocksDB for, which is why I think it’s time for something new.

To summarize, I would like an embedded database with the following features:

  • A reasonable type system and language semantics that resemble SQL only closely enough for people to port their understanding of the relational model over.
  • Bounded memory use and linear-time queries. Any operations that this can’t be guaranteed for are not allowed.
  • Asynchronous schema change support. Both logical (adding columns) and physical (adding or removing secondary indexes).
  • No query planner, plans are executed as they are written. I could soften on this one for very reliable transformations like removing redundant predicates, which are extremely important if you do any kind of programmatic query construction, but it should be basically possible to tell how the query will run by looking at it.
  • The ability to switch between row- and column-based layouts.

Things like this have been tried, but I think their time has come again. No more KV.

Source Link


Keep your files stored safely and securely with the SanDisk 2TB Extreme Portable SSD. With over 69,505 ratings and an impressive 4.6 out of 5 stars, this product has been purchased over 8K+ times in the past month. At only $129.99, this Amazon’s Choice product is a must-have for secure file storage.

Help keep private content private with the included password protection featuring 256-bit AES hardware encryption. Order now for just $129.99 on Amazon!


Start your free Amazon Prime trial
today and unlock unlimited streaming and more!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!

BITCOIN

Bitcoin Logo

Bitcoin QR Code

bc1qlszw7elx2qahjwvaryh0tkgg8y68enw30gpvge

Scan the QR code with your crypto wallet app

DOGECOIN

Dogecoin Logo

Dogecoin QR Code

D64GwvvYQxFXYyan3oQCrmWfidf6T3JpBA

Scan the QR code with your crypto wallet app

ETHEREUM

Ethereum Logo

Ethereum QR Code

0xe9BC980DF3d985730dA827996B43E4A62CCBAA7a

Scan the QR code with your crypto wallet app

Please read the Privacy and Security Disclaimer on how Techcratic handles your support.

Disclaimer: As an Amazon Associate, Techcratic may earn from qualifying purchases.

Tags: Hacker News
Share162Share28ShareShare4ShareTweet101
Previous Post

Customize Adaptive Audio on AirPods 4 and AirPods Pro 2

Next Post

Nvidia’s Role in the Next Generation of Humanoid Robots

Hacker News

Hacker News

Stay updated with Hacker News, where technology meets entrepreneurial spirit. Get the latest on tech trends, startup news, and discussions from the tech community. Read the latest updates here at Techcratic.

Related Posts

Welcome to the Two Towers Mud!
Hacker News

Welcome to the Two Towers Mud!

July 8, 2025
1.3k
BladedFeline Exploits Whisper and PrimeCache to Breach IIS and Microsoft Exchange Servers
Hacker News

BladedFeline Exploits Whisper and PrimeCache to Breach IIS and Microsoft Exchange Servers

July 7, 2025
1.3k
Introducing Tyr, a new Rust DRM driver
Hacker News

Introducing Tyr, a new Rust DRM driver

July 7, 2025
1.3k
New Slopsquatting Attack Exploits Coding Agent Workflows to Deliver Malware
Hacker News

New Slopsquatting Attack Exploits Coding Agent Workflows to Deliver Malware

July 7, 2025
1.3k
AiLock Ransomware Emerges with Hybrid Encryption Tactics: ChaCha20 Meets NTRUEncrypt
Hacker News

AiLock Ransomware Emerges with Hybrid Encryption Tactics: ChaCha20 Meets NTRUEncrypt

July 7, 2025
1.3k
joelburget/microjax: A tiny autograd engine with a Jax-like API
Hacker News

joelburget/microjax: A tiny autograd engine with a Jax-like API

July 7, 2025
1.3k
New Linux EDR Evasion Tool Exploits io_uring Kernel Feature
Hacker News

New Linux EDR Evasion Tool Exploits io_uring Kernel Feature

July 7, 2025
1.3k
The Era of Full-Stack Chip Designers
Hacker News

The Era of Full-Stack Chip Designers

July 7, 2025
1.3k
Load More
Next Post
Nvidia’s Role in the Next Generation of Humanoid Robots

Nvidia's Role in the Next Generation of Humanoid Robots

Your Tech Resources

  • 30 Second Tech ™
  • AI
  • App Zone ™
  • Apple
  • Ars Technica
  • CNET
  • ComputerWorld
  • Crypto News
  • Cybersecurity
  • Endgadget
  • Forbes
  • Fossbytes
  • Gaming
  • GeekWire
  • Gizmodo
  • Google News
  • Hacker News
  • Harvard Tech
  • I Like Cats ™
  • I Like Dogs ™
  • LifeHacker
  • MacRumors
  • Macworld
  • Mashable
  • Microsoft
  • MIT Tech
  • PC World
  • Photofocus
  • Physics
  • Random Tech
  • Retro Rewind ™
  • Robot Report
  • SiliconANGLE
  • SlashGear
  • Smartphone
  • StackSocial
  • Tech Art
  • Tech Careers
  • Tech Deals
  • Techcratic ™
  • TechCrunch
  • Techdirt
  • TechRepublic
  • Techs Got To Eat ™
  • TechSpot
  • Tesla
  • The Verge
  • TNW
  • Trusted Reviews
  • UFO
  • VentureBeat
  • Visual Capitalist
  • Wired
  • ZDNet

Tech News

  • 30 Second Tech ™
  • AI
  • Apple Insider
  • Ars Technica
  • CNET
  • ComputerWorld
  • Crypto News
  • Cybersecurity
  • Endgadget
  • ExtremeTech
  • Fossbytes
  • Gaming
  • GeekWire
  • Gizmodo

Tech News

  • Harvard Tech
  • MacRumors
  • Macworld
  • Mashable
  • Microsoft
  • MIT Tech
  • Physics
  • PC World
  • Random Tech
  • Retro Rewind ™
  • SiliconANGLE
  • SlashGear
  • Smartphone
  • StackSocial
  • Tech Careers

Tech News​

  • Tech Art
  • TechCrunch
  • Techdirt
  • TechRepublic
  • Techs Got To Eat ™
  • TechSpot
  • Tesla
  • The Verge
  • TNW
  • Trusted Reviews
  • UFO
  • VentureBeat
  • Visual Capitalist
  • Wired
  • ZDNet

Site Links

  • About Techcratic
  • Affiliate Disclaimer
  • Affiliate Link Policy
  • Contact Techcratic
  • Dealors Discount Store
  • Privacy and Security Disclaimer
  • Privacy Policy
  • RSS Feed
  • Site Map
  • Support Techcratic
  • Techcratic
  • Tech Deals
  • TOS
  • 𝕏
Click For A Secret Deal

OnePlus introduces Buds 4 and Watch 3 43mm

Welcoming Furry Joy!! Unleash Excitement as Adorable Puppies Find Home Sweet Home

iPhone 17 Air rumor: New sky blue color

Police Checkpoint |As Dusk Falls Ep.25

Buying a new Mac? Here’s why you should switch from a MacBook to a desktop

The best deals from Prime Day 2025

Techcratic – Your All In One Tech Hub © 2020 – 2025
All Rights Reserved
∞

No Result
View All Result
  • 30 Second Tech ™
  • AI
  • App Zone ™
  • Apple
  • Ars Technica
  • CNET
  • Crypto News
  • Cybersecurity
  • Endgadget
  • Gaming
  • I Like Cats ™
  • I Like Dogs ™
  • MacRumors
  • Macworld
  • Tech Deals
  • Techcratic ™
  • Techs Got To Eat ™
  • Tesla
  • UFO
  • Wired