Storage Operations

Purpose

This guide shows you how to get started with storage operations in Synapse SDK. You’ll learn the fundamentals of uploading data, managing data sets, and retrieving stored content.

Audience: Developers new to Synapse SDK storage capabilities Time to complete: 10-15 minutes Prerequisites: Basic TypeScript knowledge, Synapse SDK installed

This guide explains the core storage concepts and provides examples of how to use the Synapse SDK to store, retrieve, and manage data on Filecoin On-Chain Cloud.

Key Concepts

Data Set: A logical container of pieces stored with one provider. When a data set is created, a payment rail is established with that provider. All pieces in the data set share this single payment rail and are verified together via PDP proofs.

PieceCID: Content-addressed identifier for your data (format: bafkzcib...). Automatically calculated during upload and used to retrieve data from any provider.

Metadata: Optional key-value pairs for organization:

• Data Set Metadata: Max 10 keys (e.g., project, environment)
• Piece Metadata: Max 5 keys per piece (e.g., filename, contentType)

Storage Manager: The main entry point for storage operations. Handles provider selection, data set management, and provides downloads from any provider (provider-agnostic) using the StorageContext.

Storage Context: A connection to a specific storage provider and data set. Created explicitly for fine-grained control or automatically by StorageManager. Enables uploads and downloads with the specific storage provider.

Storage Approaches

The SDK offers two ways to work with storage operations:

Approach	Who It’s For	What SDK Handles	When to Use
Auto-Managed	Most developers	Provider selection, data set creation, management	Getting started, simple apps, quick prototypes
Explicit Control	Advanced users	Nothing - you control everything	Batch operations, specific providers, cost optimization

Recommendation: Start with auto-managed, then explore explicit control only if needed.

Example 1: Quick Start (Auto-Managed)

Upload and download data with zero configuration - SDK automatically selects a provider and manages the data set:

const data = new Uint8Array([1, 2, 3, 4, 5]);

const result = await synapse.storage.upload(data);
const downloaded = await synapse.storage.download(result.pieceCid);

console.log("Uploaded:", result.pieceCid);
console.log("Downloaded:", downloaded.length, "bytes");

Example 2: With Metadata (Auto-Managed)

Add metadata to organize uploads and enable faster data set reuse - SDK will reuse any existing data set matching the metadata:

const context = await synapse.storage.createContext({
  metadata: {
    Application: "My DApp",
    Version: "1.0.0",
    Category: "Documents",
  },
});

const result = await synapse.storage.upload(data, { contexts: [context] });
console.log("Uploaded:", result.pieceCid);

Need More Control?

The examples above use auto-managed storage where the SDK handles provider selection and data set creation.

For advanced use cases like batch uploads, custom callbacks, or explicit provider selection, see the Storage Context Guide.

Upload Result

When you call upload(), the SDK selects storage providers, uploads your data, and commits it on-chain. By default it stores 2 copies on separate providers for redundancy. The result tells you exactly what happened:

const result = await synapse.storage.upload(data)

// The PieceCID identifies your data across all providers
console.log("PieceCID:", result.pieceCid)

// Each copy is committed on-chain with its own provider and data set
for (const copy of result.copies) {
  console.log(`Copy on provider ${copy.providerId}, dataset ${copy.dataSetId}`)
}

// If any provider failed, it appears here (the upload still succeeded on others)
for (const failure of result.failures) {
  console.warn(`Provider ${failure.providerId} failed: ${failure.error}`)
}

Key points:

• result.copies — each entry is a confirmed on-chain copy of your data. If you requested 2 copies and both succeeded, you get 2 entries.
• result.failures — providers that failed during the upload. Empty when everything worked. The upload still returns a result as long as at least one copy succeeded.
• If the upload can’t store your data at all, it throws (StoreError). If data was stored but couldn’t be committed on any provider, it throws (CommitError). Both are safe to retry.

You can request more copies with { count: 3 } for additional redundancy.

Data Set Management

When You Need This

These APIs are useful when you want to inspect existing data sets, query stored pieces, or retrieve metadata. For basic upload/download, you don’t need these.

Data sets are automatically created during your first upload to a provider. For explicit management of data sets, use these operations:

When You Need Explicit Data Sets:

• Uploading many files to same provider
• Want consistent provider for your application
• Need to track costs per data set
• Building batch upload workflows

Getting all data sets

Retrieve all data sets owned by your account to inspect piece counts, CDN status, and metadata:

const dataSets = await synapse.storage.findDataSets();

for (const ds of dataSets) {
  console.log(`Dataset ${ds.pdpVerifierDataSetId}:`, {
    live: ds.isLive,
    cdn: ds.withCDN,
    pieces: ds.activePieceCount,
    metadata: ds.metadata
  });
}

Getting data set pieces

List all pieces stored in a specific data set by iterating through the context:

const context = await synapse.storage.createContext({ dataSetId });

const pieces = [];
for await (const piece of context.getPieces()) {
  pieces.push(piece);
}
console.log(`Found ${pieces.length} pieces`);

Getting data set size

Calculate total storage size by summing piece sizes extracted from PieceCIDs:

const pdpVerifier = PDPVerifier.create();

const leafCount = await pdpVerifier.getDataSetLeafCount(dataSetId);
const sizeInBytes = leafCount * 32n; // Each leaf is 32 bytes
console.log(`Data set size: ${sizeInBytes} bytes`);

Getting a data set piece metadata

Access custom metadata attached to individual pieces for organization and filtering:

const warmStorage =  WarmStorageService.create();

const metadata = await warmStorage.getPieceMetadata(dataSetId, piece.pieceId);
console.log("Piece metadata:", metadata);

Getting the size of a specific piece

Calculate size of a specific piece by extracting the size from the PieceCID:

import { getSizeFromPieceCID } from "@filoz/synapse-sdk/piece";

const size = getSizeFromPieceCID(pieceCid);
console.log(`Piece size: ${size} bytes`);

Storage Information

Query service-wide pricing, available providers, and network parameters:

const info = await synapse.getStorageInfo();
console.log("Price/TiB/month:", info.pricing.noCDN.perTiBPerMonth);
console.log("Providers:", info.providers.length);

const providerInfo = await synapse.getProviderInfo("0x...");
console.log("PDP URL:", providerInfo.pdp.serviceURL);

Next Steps

Ready to explore more? Here’s your learning path:

• Advanced Operations → Learn about batch uploads, lifecycle management, and download strategies. For developers building production applications with specific provider requirements.

• Plan Storage Costs → Calculate your monthly costs and understand funding requirements. Use the quick calculator to estimate costs in under 5 minutes.

• Payment Management → Manage deposits, approvals, and payment rails. Required before your first upload.