Excellent breakdown of SST internals. The connection between 4KB page sizes and block-aligned reads is something I see glossed over constanty in higher-level database discussions. The experiment showing 1KB vs 4KB having identical latency really drives home why spatial locality is so criticial. One thing that might be worth expanding on is how prefix compression affects the bloom filter FPR in practice since encoded keys shift the distribtuion.
Interesting point thanks for the comment! I hadn't previously considered the effect of prefix compression on bloom filter FPRs. I know that in theory it might cause some issues with higher collision rates with the hash functions but I feel like a good enough hash function should be able to handle this gracefully enough. Have you had experience in production where this is a problem?
Excellent breakdown of SST internals. The connection between 4KB page sizes and block-aligned reads is something I see glossed over constanty in higher-level database discussions. The experiment showing 1KB vs 4KB having identical latency really drives home why spatial locality is so criticial. One thing that might be worth expanding on is how prefix compression affects the bloom filter FPR in practice since encoded keys shift the distribtuion.
Interesting point thanks for the comment! I hadn't previously considered the effect of prefix compression on bloom filter FPRs. I know that in theory it might cause some issues with higher collision rates with the hash functions but I feel like a good enough hash function should be able to handle this gracefully enough. Have you had experience in production where this is a problem?