feat(fibre): implement Fibre Module by vgonkivs · Pull Request #4892 · celestiaorg/celestia-node

vgonkivs · 2026-03-25T16:59:36Z

Resolves https://linear.app/celestia/issue/PROTOCO-1332/implement-fibre-module

Wondertan

There are two competing fibre blob submission paths, and one of them is incomplete

modfibre.Upload

The newly introduced API, however, is incomplete as it doesn't send the PFF transactions. The comment mentions that users should use modblob.SubmitFibreBlob to send TXs; however, doing so requires reuploading the entire blob over the API, and modblob.SubmitFibreBlob reuploads it again.

Essentially, there is no way for fibre users of Upload to pay for their uploads

modblob.SubmitFibreBlob

The purpose of this method is unclear. It duplicates the entire fibre blob submission on the old blob service, however, the intention was clearly to introduce a new API as seen in the new fibre module, so why duplicate? If the intention was to allow sending PFFs through this module, then it is unclear why blob module would be responsible for that and not fibre.

Too many layers of indirections

With this PR we have:

modfibre.Module
fibre.Client
fibre.client
txclient fibre methods
appfibre.Client

This is extremely messy and a pure pain to review. We can easily squash a bunch of them with no repercussions into:

modfibre.Module
fibre.Service
appfibre.Client.

Besides, for whatever reason, out of all those layers, the txclient turned out to be responsible for actually uploading. TxClient, Carl! It is supposed to send transactions and not call appfibre.Client.Upload.

Subscriptions

There is no way to subscribe for blobs from fibre in the order they'be landed through consensus.

nodebuilder/fibre/fibre.go

fibre/metrics.go

state/txclient/fibre.go

vgonkivs · 2026-04-02T10:37:25Z

This PR was done in accordance to the ADR, made by @cmwaters.

There are two competing fibre blob submission paths, and one of them is incomplete
modfibre.Upload
The newly introduced API, however, is incomplete as it doesn't send the PFF transactions. The comment mentions that users should use modblob.SubmitFibreBlob to send TXs; however, doing so requires reuploading the entire blob over the API, and modblob.SubmitFibreBlob reuploads it again.

Link1

Link2

Subscriptions
There is no way to subscribe for blobs from fibre in the order they'be landed through consensus.

Auto-fetching full fibre data from FSPs needs design work first. Feel free to open an ADR update with a proposal

Wondertan · 2026-04-02T11:03:22Z

@vgonkivs, I acknowledge that the PR implements the ADR. However, it does not change the fact that users can't pay for the uploads they make and we should fix this.

Auto-fetching full fibre data from FSPs needs design work first. Feel free to open an ADR update with a proposal

The subscription does not imply listening for data from FSPs, but listening to new fibre-blobs in the square and fetching respective fibre blobs by commitment.

vgonkivs · 2026-04-02T13:21:44Z

Feel free to open an ADR update with a proposal

vgonkivs · 2026-04-02T13:26:04Z

However, it does not change the fact that users can't pay for the uploads they make and we should fix this.

https://github.com/celestiaorg/celestia-node/pull/4892/changes#diff-575205cc93599bc2a9d28e62e576697e9fcf39733970a61aeebfade20493f1dbR50

Wondertan · 2026-04-02T13:38:12Z

https://github.com/celestiaorg/celestia-node/pull/4892/changes#diff-575205cc93599bc2a9d28e62e576697e9fcf39733970a61aeebfade20493f1dbR50

What about this link?

Wondertan · 2026-04-02T13:40:26Z

Feel free to open an ADR update with a proposal

Nothing is stopping us from modifying the ADR in this PR. That's a normal feedback process where during implementation issues are discovered and spec/adr is updated accordingly.

vgonkivs · 2026-04-03T14:57:55Z

There is no final decision yet on the upload, so no changes were made to this part.

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

fibre/service.go

Wondertan

last pass, mostly nits and questions

P.S. I haven't reviewed the AccountClient API from a user perspective to determine whether the added endpoints make sense and cover all use cases. The case with Upload completely disincentivized me from reviewing user flow completeness. While the point was correct, it created more tension than good because of the communication style. I trust you guys can review and communicate such things better.

Wondertan · 2026-04-10T13:07:08Z

Makefile

 test-unit:
 	@echo "--> Running unit tests"
-	@go test $(VERBOSE) -covermode=atomic -coverprofile=coverage.txt `go list ./... | grep -v nodebuilder/tests` $(LOG_AND_FILTER)
+	@go test $(VERBOSE) -tags=fibre -covermode=atomic -coverprofile=coverage.txt `go list ./... | grep -v nodebuilder/tests` $(LOG_AND_FILTER)


do we need this tag so that app picks the fibre app?

w/o the tag all tests that are using app node will be constantly failing. It's a temp solution.

Wondertan · 2026-04-10T13:09:58Z

nodebuilder/fibre/types.go

+type SubmitResult struct {
+	// BlobID is the Fibre blob identifier for the submitted blob.


just an idea to make SubmitResult embed the UploadResult. This will imply that Submit operation does Upload + some stuff which it is and deduplication.

Wondertan · 2026-04-10T13:13:10Z

nodebuilder/fibre/fibre.go

+	// It encodes the blob, constructs a payment promise, uploads encoded rows to FSPs,
+	// and aggregates validator availability signatures. It does NOT submit MsgPayForFibre on-chain.
+	// Use blob.SubmitFibreBlob for the full submit flow.
+	Upload(_ context.Context, _ libshare.Namespace, _ []byte, _ *txclient.TxConfig) (*UploadResult, error)


nit: not sure why they are all underscore, would be helpful to name bytes as data for example

Wondertan · 2026-04-10T13:15:17Z

nodebuilder/fibre/fibre.go

+	// It encodes the blob, constructs a payment promise, uploads encoded rows to FSPs,
+	// and aggregates validator availability signatures. It does NOT submit MsgPayForFibre on-chain.
+	// Use blob.SubmitFibreBlob for the full submit flow.
+	Upload(_ context.Context, _ libshare.Namespace, _ []byte, _ *txclient.TxConfig) (*UploadResult, error)


If it doesn't send tx, why does it expect txclient? To future proof?

Wondertan · 2026-04-10T13:17:49Z

nodebuilder/fibre/fibre.go

+	// It encodes the blob, constructs a payment promise, uploads encoded rows to FSPs,
+	// and aggregates validator availability signatures. It does NOT submit MsgPayForFibre on-chain.
+	// Use blob.SubmitFibreBlob for the full submit flow.
+	Upload(_ context.Context, _ libshare.Namespace, _ []byte, _ *txclient.TxConfig) (*UploadResult, error)


I would mention that we send v0 fibre blob as we currently do no support specifying which version

Wondertan · 2026-04-10T13:18:26Z

nodebuilder/fibre/fibre.go

+	Upload(_ context.Context, _ libshare.Namespace, _ []byte, _ *txclient.TxConfig) (*UploadResult, error)
+	// Get retrieves a Fibre blob from FSPs by blobID.
+	// It reconstructs the original blob data from the encoded rows stored off-chain.
+	Get(ctx context.Context, blobID []byte) (*GetBlobResult, error)


Why not a specific type for blobID?

to avoid external types where it's possible (even they are type aliases)

Wondertan · 2026-04-10T13:19:14Z

fibre/service.go

+	if err != nil {
+		return nil, fmt.Errorf("invalid blob ID: %w", err)
+	}
+	log.Debugw("downloading fibre blob", "commitment", blbID.Commitment())


This should move to Download

will move and then remove it from here

Wondertan · 2026-04-10T13:22:25Z

nodebuilder/fibre/fibre.go

+	Upload(_ context.Context, _ libshare.Namespace, _ []byte, _ *txclient.TxConfig) (*UploadResult, error)
+	// Get retrieves a Fibre blob from FSPs by blobID.
+	// It reconstructs the original blob data from the encoded rows stored off-chain.
+	Get(ctx context.Context, blobID []byte) (*GetBlobResult, error)


Calling it Download like the client does would be slightly easier for us terminologically. The Get by itself doesn't do anything else but calls Download, so not much value giving it another name. Then if we refer to it as Download we know its equivalent to client's download

Wondertan · 2026-04-10T13:24:22Z

fibre/service.go

+	ns libshare.Namespace,
+	data []byte,
+	_ *txclient.TxConfig,
+) (_ *appfibre.SignedPaymentPromise, _ appfibre.BlobID, err error) {


nit: you can technically retrieve blobId from PaymentPromise, we could even add a BlobID method to it. BlobID is Comm + BlobVer and both are fields of PP.

For the external api in the module, it is still makes sense to keep BlobID as independent field, but for the intermal service we could deduplicate things amd debloat function signature

Wondertan · 2026-04-10T13:32:35Z

fibre/account.go

+}
+
+func (a *AccountClient) QueryEscrowAccount(ctx context.Context, signer string) (_ *EscrowAccount, err error) {
+	if a == nil {


a bit confused how is it possible for AccountClient to be nil but not for the entire service. When grpc address is not supplied? But then we can't still send PFF txs right? Or do you want to keep it for reads? We may want to protect the Upload/Submit with ErrNoClient as well

It's more a guard rather than smth specific. But I agree, that we should keep consistency. Will add the same condition to Upload/Submit

vgonkivs self-assigned this Mar 25, 2026

vgonkivs requested a review from a team as a code owner March 25, 2026 16:59

vgonkivs requested a review from walldiss March 25, 2026 16:59

This comment was marked as resolved.

Sign in to view

vgonkivs force-pushed the add_fibre_module branch 3 times, most recently from 53ba2fa to cbe0341 Compare March 25, 2026 17:18

This comment was marked as resolved.

Sign in to view

vgonkivs force-pushed the add_fibre_module branch from cbe0341 to 1238fe5 Compare March 26, 2026 09:07

This comment was marked as resolved.

Sign in to view

vgonkivs force-pushed the add_fibre_module branch from 1238fe5 to 241b13a Compare March 26, 2026 09:29

This comment was marked as resolved.

Sign in to view

vgonkivs force-pushed the add_fibre_module branch from 241b13a to 25efce9 Compare March 26, 2026 09:43

This comment was marked as resolved.

Sign in to view

vgonkivs force-pushed the add_fibre_module branch from 25efce9 to 42cd176 Compare March 26, 2026 09:59

This comment was marked as resolved.

Sign in to view

vgonkivs force-pushed the add_fibre_module branch from 42cd176 to 58a1c46 Compare March 26, 2026 10:24

This comment was marked as resolved.

Sign in to view

vgonkivs force-pushed the add_fibre_module branch from 58a1c46 to 1f4b918 Compare March 26, 2026 12:29

This comment was marked as resolved.

Sign in to view

Wondertan requested changes Apr 1, 2026

View reviewed changes

nodebuilder/fibre/fibre.go Outdated Show resolved Hide resolved

nodebuilder/fibre/fibre.go Outdated Show resolved Hide resolved

Wondertan reviewed Apr 1, 2026

View reviewed changes

fibre/metrics.go Outdated Show resolved Hide resolved

state/txclient/fibre.go Outdated Show resolved Hide resolved

vgonkivs requested a review from Wondertan April 3, 2026 14:57

vgonkivs force-pushed the add_fibre_module branch from 74da0df to ccf7be5 Compare April 8, 2026 11:56

This comment was marked as resolved.

Sign in to view

vgonkivs force-pushed the add_fibre_module branch from 2207eb5 to a02e7c8 Compare April 8, 2026 13:19

vgonkivs and others added 14 commits April 9, 2026 11:01

feat(fibre): add fibre types

f57161d

feat(fibre): add fibre client

56ab408

feat(fibre): add escrow account client

e8fe112

feat(fibre): add metrics for blob and account

e21ca02

feat(state): add fibre methods to TxClient

7309e87

feat(nodebuilder): add fibre module

67e9ec2

feat(blob): add SubmitFibreBlob to blob service

6ddda92

feat(api): add fibre to RPC and read clients

6e1bd26

chore: add fibre to misspell exceptions

050be2b

fix(state): fix fibre/txClient start

a6ba582

apply suggestions

3115def

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

refactoring

61faca2

remove submission path from blob service

023c499

apply suggestions left by @Wondertan

8bb3c83

vgonkivs force-pushed the add_fibre_module branch from a02e7c8 to 2feab33 Compare April 9, 2026 08:03

reuse keyring keyName for fibre

ab049b4

vgonkivs force-pushed the add_fibre_module branch from 2feab33 to ab049b4 Compare April 9, 2026 11:29

vgonkivs requested a review from Wondertan April 9, 2026 11:31

This comment was marked as resolved.

Sign in to view

Wondertan requested changes Apr 9, 2026

View reviewed changes

fibre/service.go Outdated Show resolved Hide resolved

vgonkivs requested a review from Wondertan April 10, 2026 09:27

This comment was marked as resolved.

Sign in to view

rework input param from commitment to blob id

988d326

vgonkivs force-pushed the add_fibre_module branch from fe7fcb6 to 988d326 Compare April 10, 2026 10:12

update ADR

4b34ddd

Wondertan reviewed Apr 10, 2026

View reviewed changes

		type SubmitResult struct {
		// BlobID is the Fibre blob identifier for the submitted blob.

Conversation

vgonkivs commented Mar 25, 2026 • edited by devin-ai-integration bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Wondertan left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

There are two competing fibre blob submission paths, and one of them is incomplete

Too many layers of indirections

Subscriptions

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vgonkivs commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Wondertan commented Apr 2, 2026

Uh oh!

vgonkivs commented Apr 2, 2026

Uh oh!

vgonkivs commented Apr 2, 2026

Uh oh!

Wondertan commented Apr 2, 2026

Uh oh!

Wondertan commented Apr 2, 2026

Uh oh!

vgonkivs commented Apr 3, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Wondertan left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

vgonkivs commented Mar 25, 2026 •

edited by devin-ai-integration bot

Loading

Wondertan left a comment •

edited

Loading

vgonkivs commented Apr 2, 2026 •

edited

Loading

Wondertan left a comment •

edited

Loading