Skip to content

Commit 1517b34

Browse files
fix(sqlite-storage): fix snapshot group_id encoding mismatch and switch MLS codec to postcard (marmot-protocol#179)
* fix(sqlite-storage): use JsonCodec-serialized keys for OpenMLS snapshot queries The snapshot/restore code used group_id.as_slice() (raw bytes) to query OpenMLS tables, but the StorageProvider writes group_id via JsonCodec::serialize() (JSON-encoded bytes). The WHERE clauses never matched, so snapshots silently captured zero OpenMLS rows and rollbacks left MLS cryptographic state untouched — creating a metadata/crypto split-brain where MDK thought it was at epoch N but the MLS engine had epoch N+1 keys. Add 9 failing-then-fixed tests to mdk-sqlite-storage and 4 regression guard tests to mdk-memory-storage that exercise the MLS StorageProvider code path for snapshot capture and rollback. * refactor(storage): replace JSON codec with postcard binary codec for MLS storage Switch MLS storage serialization from serde_json to postcard, a compact binary serde format. A 32-byte group_id now serializes to ~33 bytes instead of ~130 bytes with JSON, reducing storage overhead and index size for database lookups. Rename JsonCodec to MlsCodec across the codebase. This is a breaking change for any persisted MLS data — existing databases are incompatible and must be recreated. * docs: add changelog entries and fix import placement Add changelog entries for PR marmot-protocol#179 across all four affected crates. Move MdkStorageProvider import to module scope in memory storage tests per STYLE.md import placement rules.
1 parent d6777f7 commit 1517b34

13 files changed

Lines changed: 1193 additions & 106 deletions

File tree

Cargo.lock

Lines changed: 34 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ nostr = { version = "0.44", default-features = false }
2929
# Serialization
3030
serde = { version = "1.0", default-features = false }
3131
serde_json = { version = "1.0", default-features = false }
32+
postcard = { version = "1.1", default-features = false }
3233
tls_codec = "0.4"
3334

3435
# Cryptography

crates/mdk-core/CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525

2626
### Breaking changes
2727

28+
- **`hash_ref_bytes` format change**: The `hash_ref_bytes` returned by `create_key_package_for_event` and consumed by `delete_key_package_from_storage_by_hash_ref` are now postcard-encoded instead of JSON-encoded. Previously stored hash_ref bytes are incompatible and cannot be deserialized. ([#179](https://github.com/marmot-protocol/mdk/pull/179))
2829
- **`create_key_package_for_event` Return Type Change**: The return type changed from `(String, Vec<Tag>)` to `(String, Vec<Tag>, Vec<u8>)`. The third element is the serialized hash_ref of the key package, computed atomically during creation. This enables callers to track key packages for lifecycle management (publish → consume → cleanup) without needing to re-parse the package. Callers that don't need the hash_ref can destructure with `_`. ([#178](https://github.com/marmot-protocol/mdk/pull/178))
2930
- **`create_key_package_for_event` Return Type Change**: The return type changed from `(String, [Tag; 7])` to `(String, Vec<Tag>)`. Most code patterns (iteration, indexing) continue to work unchanged. This change was necessary because the protected tag is now optional. ([#173](https://github.com/marmot-protocol/mdk/pull/173), related: [#168](https://github.com/marmot-protocol/mdk/issues/168))
3031
- **`create_key_package_for_event` No Longer Adds Protected Tag**: The `create_key_package_for_event()` function no longer adds the NIP-70 protected tag (`["-"]`) by default. This is a behavioral change - existing code that relied on the protected tag being present will now produce key packages without it. Key packages can now be republished by third parties to any relay. This improves relay compatibility since many popular relays (Damus, Primal, nos.lol) reject protected events outright. For users who need the protected tag, use the new `create_key_package_for_event_with_options()` function with `protected: true`. ([#173](https://github.com/marmot-protocol/mdk/pull/173), related: [#168](https://github.com/marmot-protocol/mdk/issues/168))

crates/mdk-core/src/key_packages.rs

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
//! MDK Key Packages
22
33
use mdk_storage_traits::MdkStorageProvider;
4-
use mdk_storage_traits::mls_codec::JsonCodec;
4+
use mdk_storage_traits::mls_codec::MlsCodec;
55
use nostr::{Event, Kind, PublicKey, RelayUrl, Tag, TagKind};
66
use openmls::ciphersuite::hash_ref::HashReference;
77
use openmls::key_packages::KeyPackage;
@@ -138,7 +138,7 @@ where
138138
let hash_ref = key_package_bundle
139139
.key_package()
140140
.hash_ref(self.provider.crypto())?;
141-
let hash_ref_bytes = JsonCodec::serialize(&hash_ref)
141+
let hash_ref_bytes = MlsCodec::serialize(&hash_ref)
142142
.map_err(|e| Error::Provider(format!("Failed to serialize hash_ref: {}", e)))?;
143143

144144
let key_package_serialized = key_package_bundle.key_package().tls_serialize_detached()?;
@@ -529,7 +529,7 @@ where
529529
&self,
530530
hash_ref_bytes: &[u8],
531531
) -> Result<(), Error> {
532-
let hash_ref: HashReference = JsonCodec::deserialize(hash_ref_bytes)
532+
let hash_ref: HashReference = MlsCodec::deserialize(hash_ref_bytes)
533533
.map_err(|e| Error::Provider(format!("Failed to deserialize hash_ref: {}", e)))?;
534534

535535
self.provider

crates/mdk-memory-storage/CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@
2525

2626
## Unreleased
2727

28+
### Breaking changes
29+
30+
- **MLS codec switched from JSON to postcard**: MLS storage serialization now uses `MlsCodec` (postcard binary format) instead of the removed `JsonCodec` (serde_json). In-memory data is ephemeral so no migration is needed, but the serialized byte format has changed. ([#179](https://github.com/marmot-protocol/mdk/pull/179))
31+
2832
### Added
2933

3034
- **Custom Message Sort Order**: `messages()` now respects the `sort_order` field in `Pagination`, supporting both `CreatedAtFirst` (default) and `ProcessedAtFirst` orderings. ([#171](https://github.com/marmot-protocol/mdk/pull/171))

0 commit comments

Comments
 (0)