Skip to content

feat: Add in-memory blockstore#160

Merged
zvolin merged 12 commits into
celestiaorg:mainfrom
fl0rek:feat/in-memory-blockstore
Dec 15, 2023
Merged

feat: Add in-memory blockstore#160
zvolin merged 12 commits into
celestiaorg:mainfrom
fl0rek:feat/in-memory-blockstore

Conversation

@fl0rek
Copy link
Copy Markdown
Contributor

@fl0rek fl0rek commented Dec 6, 2023

closes #162

@fl0rek fl0rek requested review from oblique and zvolin and removed request for oblique December 6, 2023 08:20
@fl0rek fl0rek force-pushed the feat/in-memory-blockstore branch from 37b286c to fd9e451 Compare December 6, 2023 08:27
@fl0rek fl0rek requested a review from oblique December 8, 2023 08:34
Comment thread blockstore/src/lib.rs
Copy link
Copy Markdown
Contributor

@oblique oblique left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@fl0rek fl0rek marked this pull request as draft December 12, 2023 10:20
@fl0rek fl0rek force-pushed the feat/in-memory-blockstore branch 2 times, most recently from 0267613 to 3f0e1dc Compare December 12, 2023 20:54
@fl0rek fl0rek force-pushed the feat/in-memory-blockstore branch from 3f0e1dc to b0ba48c Compare December 13, 2023 08:05
@fl0rek fl0rek marked this pull request as ready for review December 13, 2023 09:00
@fl0rek fl0rek requested a review from oblique December 13, 2023 09:00
Comment thread blockstore/src/lib.rs
@fl0rek fl0rek force-pushed the feat/in-memory-blockstore branch from ec35957 to 1af0e1c Compare December 13, 2023 12:23
@fl0rek fl0rek force-pushed the feat/in-memory-blockstore branch from 1af0e1c to be8d776 Compare December 13, 2023 12:31
@oblique
Copy link
Copy Markdown
Contributor

oblique commented Dec 14, 2023

I'm confused. Why Block couldn't be a simpler? Like this:

pub trait Block<const S: usize>: Sync + Send {
    fn cid(&self) -> CidGeneric<S>;
    fn data(&self) -> &[u8];
}

@oblique
Copy link
Copy Markdown
Contributor

oblique commented Dec 14, 2023

I tried this and works:

blockstore/src/lib.rs

pub trait Blockstore {
    // ....

    async fn put_many<const S: usize, B, I>(&self, blocks: I) -> Result<()>
    where
        B: Block<S>,
        I: IntoIterator<Item = B> + Send,
        <I as IntoIterator>::IntoIter: Send,
    {
        for b in blocks {
            let cid = b.cid()?;
            self.put_keyed(&cid, b.data()).await?;
        }
        Ok(())
    }
    
    // ...
}

pub trait Block<const S: usize>: Sync + Send {
    fn cid(&self) -> Result<CidGeneric<S>, CidError>;
    fn data(&self) -> &[u8];
}

types/src/share.rs

impl Block<NMT_ID_SIZE> for Share {
    fn cid(&self) -> Result<CidGeneric<NMT_ID_SIZE>, CidError> {
        let hasher = NamespacedSha2Hasher::with_ignore_max_ns(true);
        let digest = hasher.hash_leaf(self.as_ref()).iter().collect::<Vec<_>>();

        // size is correct, so unwrap is safe
        let hash = Multihash::wrap(NMT_CODEC, &digest).unwrap();

        Ok(CidGeneric::new_v1(NMT_MULTIHASH_CODE, hash))
    }

    fn data(&self) -> &[u8] {
        &self.data
    }
}

I prefer this simpler API for the end users

@fl0rek fl0rek force-pushed the feat/in-memory-blockstore branch from 5c545c9 to 2d2db78 Compare December 15, 2023 09:10
Comment thread blockstore/src/in_memory_blockstore.rs
Copy link
Copy Markdown
Contributor

@oblique oblique left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Copy Markdown
Collaborator

@zvolin zvolin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎸

@zvolin zvolin merged commit 129272e into celestiaorg:main Dec 15, 2023
@github-actions github-actions Bot mentioned this pull request Jan 9, 2024
@zvolin zvolin mentioned this pull request Jan 12, 2024
@fl0rek fl0rek deleted the feat/in-memory-blockstore branch July 22, 2024 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Blockstore trait and in-memory blockstore

3 participants