Skip to content

feat: pending state journal for leader recovery - #2762

Draft
carneiro-cw wants to merge 9 commits into
mainfrom
transaction-journal
Draft

carneiro-cw wants to merge 9 commits into
mainfrom
transaction-journal

Conversation

@carneiro-cw

Copy link
Copy Markdown
Contributor

No description provided.

Persist pending-block transactions to a per-block journal file so a
SIGKILLed leader can replay and commit its pending blocks on restart.

- TransactionJournal: append/read/retire of block journals under the
  database directory, retired on drop once two newer journals exist
- recover_transaction_journal in main replays journaled blocks before
  mining starts; miner init becomes sync so interval mining starts only
  after recovery
- permanent storage resolves and anchors the db path in DATABASE_PATH
- serde derives needed to deserialize journaled types
- e2e journal-recovery test spawning a leader with an external RocksDB
  prefix
@github-actions

Copy link
Copy Markdown
Contributor

Failed to generate code suggestions for PR

CI builds with --profile ci, so the binary sits at target/ci/stratus and
the hardcoded target/debug fallback made the journal-recovery test spawn
a nonexistent path (ENOENT).
@stratus-benchmark

Copy link
Copy Markdown

Benchmark:
Run ID: bench-b8f8ce3c

Git Info:

Leader Stats:
RPS Stats: Max: 9497.00, Min: 1739.00, Avg: 7432.63, StdDev: 656.73
TPS Stats: Max: 8623.00, Min: 37.00, Avg: 7309.28, StdDev: 1047.90

Follower Stats:
Imported Blocks/s: Max: 69.00, Min: 41.00, Avg: 60.20, StdDev: 10.07
Imported Transactions/s: Max: 491845.00, Min: 302735.00, Avg: 440018.40, StdDev: 70247.25

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Flamegraphs:
Run ID: bench-ae5b5ed0

Git Info:

Flamegraphs:
idle:

write:

read:

follower:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-da6bfcb2

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 9535.00, Min: 1392.00, Avg: 7080.06, StdDev: 946.88
TPS Stats: Max: 9017.00, Min: 6.00, Avg: 6956.69, StdDev: 1157.76

Plots:

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rolling journal of stratus pending blocks for reexecution in case the miner dies

1 participant