Files
core/webapi

Web API

The web API provides access to core functions via HTTP.

It can be used by local client software to connect to the Peernet network and use functions such as share, search, and download files.

Use Considerations (when not to use it)

Do not expose this API to the internet. The web API uses the core library which uses the user's public and private key to connect to the network. It shall only run on a loopback IP such as 127.0.0.1 or ::1. Special HTTP headers (including the Access-Control headers) are intentionally not set.

The API is unauthenticated and provides direct access to the users blockchain.

Deployment

The API must be initialized and started before use.

webapi.Start([]string{"127.0.0.1:112"}, false, "", "", 10*time.Second, 10*time.Second)

// To register an additional API endpoint:
webapi.Router.HandleFunc("/newfunction", newFunction).Methods("GET")

Available Functions

These are the functions provided by the API:

/status                         Provides current connectivity status to the network
/peer/self                      Provides information about the self peer details

/blockchain/self/header         Header of the blockchain
/blockchain/self/append         Append a block to the blockchain
/blockchain/self/read           Read a block of the blockchain
/blockchain/self/add/file       Add file to the blockchain
/blockchain/self/list/file      List all files stored on the blockchain
/blockchain/self/delete/file    Delete files from the blockchain

/profile/list                   List all profile fields
/profile/read                   Read a profile field
/profile/write                  Write profile fields
/profile/delete                 Delete profile fields

/search                         Submit a search request
/search/result                  Return search results
/search/terminate               Terminate a search

/download/start                 Start the download of a file
/download/status                Get the status of a download

/explore                        List recently shared files

/file/format                    Detect file type and format

API Documentation

Informational Functions

Status

This function informs about the current connection status of the client to the network. Additional fields will be added in the future.

Request:    GET /status
Response:   200 with JSON structure apiResponseStatus
type apiResponseStatus struct {
	Status        int  `json:"status"`        // Status code: 0 = Ok.
	IsConnected   bool `json:"isconnected"`   // Whether connected to Peernet.
	CountPeerList int  `json:"countpeerlist"` // Count of peers in the peer list. Note that this contains peers that are considered inactive, but have not yet been removed from the list.
	CountNetwork  int  `json:"countnetwork"`  // Count of total peers in the network.
	// This is usually a higher number than CountPeerList, which just represents the current number of connected peers.
	// The CountNetwork number is going to be queried from root peers which may or may not have a limited view into the network.
}

Self Information

This function returns information about the current peer.

Request:    GET /peer/self
Response:   200 with JSON structure apiResponsePeerSelf

The peer and node IDs are encoded as hex encoded strings.

type apiResponsePeerSelf struct {
	PeerID string `json:"peerid"` // Peer ID. This is derived from the public in compressed form.
	NodeID string `json:"nodeid"` // Node ID. This is the blake3 hash of the peer ID and used in the DHT.
}

Blockchain Functions

Common status codes returned by various endpoints:

const (
	BlockchainStatusOK                 = 0 // No problems in the blockchain detected.
	BlockchainStatusBlockNotFound      = 1 // Missing block in the blockchain.
	BlockchainStatusCorruptBlock       = 2 // Error block encoding
	BlockchainStatusCorruptBlockRecord = 3 // Error block record encoding
	BlockchainStatusDataNotFound       = 4 // Requested data not available in the blockchain
)

Blockchain Self Header

This function returns information about the current peer. It is not required that a peer has a blockchain. If no data is shared, there are no blocks. The blockchain does not formally have a header as each block has the same structure.

Request:    GET /blockchain/self/header
Response:   200 with JSON structure apiBlockchainHeader
type apiBlockchainHeader struct {
	PeerID  string `json:"peerid"`  // Peer ID hex encoded.
	Version uint64 `json:"version"` // Current version number of the blockchain.
	Height  uint64 `json:"height"`  // Height of the blockchain (number of blocks). If 0, no data exists.
}

Blockchain Append Block

This appends a block to the blockchain. This is a low-level function for already encoded blocks. Do not use this function. Adding invalid data to the blockchain may corrupt it which subsequently might result in blacklisting by other peers.

Request:    POST /blockchain/self/append with JSON structure apiBlockchainBlockRaw
Response:   200 with JSON structure apiBlockchainBlockStatus
type apiBlockRecordRaw struct {
	Type uint8  `json:"type"` // Record Type. See core.RecordTypeX.
	Data []byte `json:"data"` // Data according to the type.
}

type apiBlockchainBlockRaw struct {
	Records []apiBlockRecordRaw `json:"records"` // Block records in encoded raw format.
}

type apiBlockchainBlockStatus struct {
	Status  int    `json:"status"`  // Status: 0 = Success, 1 = Error invalid data
	Height  uint64 `json:"height"`  // Height of the blockchain (number of blocks).
	Version uint64 `json:"version"` // Version of the blockchain.
}

Blockchain Read Block

This reads a block of the current peer.

Request:    GET /blockchain/self/read?block=[number]
Response:   200 with JSON structure apiBlockchainBlock
type apiBlockchainBlock struct {
	Status            int                 `json:"status"`            // Status: 0 = Success, 1 = Error block not found, 2 = Error block encoding (indicates that the blockchain is corrupt)
	PeerID            string              `json:"peerid"`            // Peer ID hex encoded.
	LastBlockHash     []byte              `json:"lastblockhash"`     // Hash of the last block. Blake3.
	BlockchainVersion uint64              `json:"blockchainversion"` // Blockchain version
	Number            uint64              `json:"blocknumber"`       // Block number
	RecordsRaw        []apiBlockRecordRaw `json:"recordsraw"`        // Records raw. Successfully decoded records are parsed into the below fields.
	RecordsDecoded    []interface{}       `json:"recordsdecoded"`    // Records decoded. The encoding for each record depends on its type.
}

The array RecordsDecoded will contain any present record of the following:

  • Profile records, see apiBlockRecordProfile
  • File records, see apiBlockRecordFile

File Functions

These functions allow adding, deleting, and listing files stored on the users blockchain. Only metadata is actually stored on the blockchain.

type apiBlockRecordFile struct {
	ID          uuid.UUID         `json:"id"`          // Unique ID.
	Hash        []byte            `json:"hash"`        // Blake3 hash of the file data
	Type        uint8             `json:"type"`        // File Type. For example audio or document. See TypeX.
	Format      uint16            `json:"format"`      // File Format. This is more granular, for example PDF or Word file. See FormatX.
	Size        uint64            `json:"size"`        // Size of the file
	Folder      string            `json:"folder"`      // Folder, optional
	Name        string            `json:"name"`        // Name of the file
	Description string            `json:"description"` // Description. This is expected to be multiline and contain hashtags!
	Date        time.Time         `json:"date"`        // Date of the virtual file
	Metadata    []apiFileMetadata `json:"metadata"`    // Additional metadata.
}

type apiFileMetadata struct {
	Type uint16 `json:"type"` // See core.TagX constants.
	Name string `json:"name"` // User friendly name of the metadata type. Use the Type fields to identify the metadata as this name may change.
	// Depending on the exact type, one of the below fields is used for proper encoding:
	Text string    `json:"text"` // Text value. UTF-8 encoding.
	Blob []byte    `json:"blob"` // Binary data
	Date time.Time `json:"date"` // Date
}

Below is the list of defined metadata types. Undefined types may be used by clients, but are always mapped into the blob field.

Type Constant Encoding Info
0 TagName Text Mapped into Name field. Name of file.
1 TagFolder Text Mapped into Folder field. Folder name.
2 TagDescription Text Mapped into Description field. Arbitrary description of the file. May contain hashtags.
3 TagDateShared Date Mapped into Date field. When the file was published on the blockchain. Cannot be set manually (virtual read-only tag).
4 TagDateCreated Date Date when the file was originally created.

Add File

This adds a file with the provided information to the blockchain. The date field cannot be set by the caller and is ignored.

Any file added is publicly accessible. The user should be informed about this fact in advance. The user is responsible and liable for any files shared.

Request:    POST /blockchain/self/add/file with JSON structure apiBlockAddFiles
Response:   200 with JSON structure apiBlockchainBlockStatus
type apiBlockAddFiles struct {
	Files []apiBlockRecordFile `json:"files"`
}

Example POST request to http://127.0.0.1:112/blockchain/self/add/file:

{
    "files": [{
        "id": "236de31d-f402-4389-bdd1-56463abdc309",
        "hash": "aFad3zRACbk44dsOw5sVGxYmz+Rqh8ORDcGJNqIz+Ss=",
        "type": 1,
        "format": 10,
        "size": 4,
        "name": "Test.txt",
        "folder": "sample directory/sub folder",
        "description": "",
        "metadata": []
    }]
}

Another payload example to create a new file but with a new arbitrary tag with type number 100 set to "test" and setting the metadata field "Date Created" (which is type 2 = core.TagTypeDateCreated):

{
    "files": [{
        "id": "bc32cbae-011d-4f0b-80a8-281ca93692e7",
        "hash": "aFad3zRACbk44dsOw5sVGxYmz+Rqh8ORDcGJNqIz+Ss=",
        "type": 1,
        "format": 10,
        "size": 4,
        "name": "Test.txt",
        "folder": "sample directory/sub folder",
        "description": "Example description\nThis can be any text #newfile #2021.",
        "metadata": [{
            "type": 2,
            "date": "2021-08-28T00:00:00Z"
        }]
    }]
}

List Files

This lists all files stored on the blockchain.

Request:    GET /blockchain/self/list/file
Response:   200 with JSON structure apiBlockAddFiles

Example response:

{
    "files": [{
        "id": "a59b6465-fe8c-4a61-9fcc-fe37cf711fd4",
        "hash": "aFad3zRACbk44dsOw5sVGxYmz+Rqh8ORDcGJNqIz+Ss=",
        "type": 1,
        "format": 10,
        "size": 4,
        "folder": "sample directory/sub folder",
        "name": "Test.txt",
        "description": "",
        "date": "2021-08-27T14:59:13Z",
        "metadata": []
    }, {
        "id": "bc32cbae-011d-4f0b-80a8-281ca9369211",
        "hash": "aFad3zRACbk44dsOw5sVGxYmz+Rqh8ORDcGJNqIz+Ss=",
        "type": 1,
        "format": 10,
        "size": 4,
        "folder": "sample directory/sub folder",
        "name": "Test 2.txt",
        "description": "Example description\nThis can be any text #newfile #2021.",
        "date": "2021-09-27T23:33:37Z",
        "metadata": [{
            "type": 2,
            "name": "Date Created",
            "text": "",
            "blob": null,
            "date": "2021-08-28T00:00:00Z"
        }]
    }],
    "status": 0
}

Delete File

This deletes files from the blockchain with the provided IDs. The blockchain will be refactored, which means it is recalculated without the specified files. The blockchains version number might be increased.

Request:    POST /blockchain/self/delete/file with JSON structure apiBlockAddFiles
Response:   200 with JSON structure apiBlockchainBlockStatus

Example POST request to http://127.0.0.1:112/blockchain/self/delete/file:

{
    "files": [{
        "id": "236de31d-f402-4389-bdd1-56463abdc309"
    }]
}

Example response:

{
    "status": 0,
    "height": 7,
    "version": 1
}

Profile Functions

User profile data such as the username, email address, and picture are stored on the blockchain. Profile fields are text (UTF-8) or binary encoded, depending on the type.

Note that all profile data is arbitrary and shall be considered untrusted and unverified. To establish trust, the user must load Certificates into the blockchain that validate certain data.

Below is the list of well known profile information. Clients may define additional fields. The purpose of this defined list is to provide a common mapping across different client software. Undefined types are always mapped into the blob field.

Type Constant Encoding Info
0 ProfileName Text Arbitrary username
1 ProfileEmail Text Email address
2 ProfileWebsite Text Website address
3 ProfileTwitter Text Twitter account without the @
4 ProfileYouTube Text YouTube channel URL
5 ProfileAddress Text Physical address
6 ProfilePicture Blob Profile picture

Profile List

This lists all profile fields.

Request:    GET /profile/list
Response:   200 with JSON structure apiProfileData
type apiProfileData struct {
	Fields []apiBlockRecordProfile `json:"fields"` // All fields
	Status int                     `json:"status"` // Status of the operation, only used when this structure is returned from the API. See core.BlockchainStatusX.
}

type apiBlockRecordProfile struct {
	Type uint16 `json:"type"` // See ProfileX constants.
	// Depending on the exact type, one of the below fields is used for proper encoding:
	Text string `json:"text"` // Text value. UTF-8 encoding.
	Blob []byte `json:"blob"` // Binary data
}

Example request: http://127.0.0.1:112/profile/list

Example response:

{
    "fields": [{
        "type": 0,
        "text": "Test Username 2021",
        "blob": null
    }, {
        "type": 1,
        "text": "test@example.com",
        "blob": null
    }],
    "status": 0
}

Profile Read

This reads a specific profile field. See ProfileX for recognized fields.

Request:    GET /profile/read?field=[index]
Response:   200 with JSON structure apiProfileData

Example request to read the users username: http://127.0.0.1:112/profile/read?field=0

Example response:

{
    "fields": [{
        "type": 0,
        "text": "Test Username 2021",
        "blob": null
    }],
    "status": 0
}

Profile Write

This writes profile fields. It can write multiple fields at once. See ProfileX for recognized fields.

Request:    POST /profile/write with JSON structure apiProfileData
Response:   200 with JSON structure apiBlockchainBlockStatus

Example POST request to http://127.0.0.1:112/profile/write:

{
    "fields": [{
        "type": 0,
        "text": "Test Username 2021"
    }]
}

Example response:

{
    "status": 0,
    "height": 1,
    "version": 0
}

Profile Delete

This function allows to delete profile fields. Only the type number is required. Multiple fields can be deleted at the same time.

Request:    POST /profile/delete with JSON structure apiProfileData
Response:   200 with JSON structure apiBlockchainBlockStatus

Example POST request to http://127.0.0.1:112/profile/delete (deleting the profile name):

{
    "fields": [{
        "type": 0
    }]
}

Search API

The search API provides a high-level function to search for files in Peernet. Searching is always asynchronous. /search returns an UUID which is used to loop over /search/result until the search is terminated.

The current implementation of the underlying search algorithm only searches file names. Optional filters are supported.

Submitting a Search Request

Request:    POST /search with JSON SearchRequest
Response:   200 on success with JSON SearchRequestResponse
type SearchRequest struct {
	Term        string        `json:"term"`       // Search term.
	Timeout     time.Duration `json:"timeout"`    // Timeout in seconds. 0 means default. This is the entire time the search may take. Found results are still available after this timeout.
	MaxResults  int           `json:"maxresults"` // Total number of max results. 0 means default.
	DateFrom    string        `json:"datefrom"`   // Date from, both from/to are required if set.
	DateTo      string        `json:"dateto"`     // Date to, both from/to are required if set.
	Sort        int           `json:"sort"`       // Sort order: 0 = No sorting, 1 = Relevance ASC, 2 = Relevance DESC (this should be default), 3 = Date ASC, 4 = Date DESC
	TerminateID []uuid.UUID   `json:"terminate"`  // Optional: Previous search IDs to terminate. This is if the user makes a new search from the same tab. Same as first calling /search/terminate.
	TypeFilter  int           `json:"typefilter"` // 0 = No filters used, 1 = Use file type filter, 2 = Use file format filter.
	FileType    int           `json:"filetype"`   // File type such as binary, text document etc. See core.TypeX.
	FileFormat  int           `json:"fileformat"` // File format such as PDF, Word, Ebook, etc. See core.FormatX.
}

type SearchRequestResponse struct {
	ID     uuid.UUID `json:"id"`     // ID of the search job. This is used to get the results.
	Status int       `json:"status"` // Status of the search: 0 = Success (ID valid), 1 = Invalid Term, 2 = Error Max Concurrent Searches
}

Example POST request to http://127.0.0.1:112/search:

{
    "term": "Test Search",
    "timeout": 10,
    "maxresults": 1000
}

Example response:

{
    "id": "ac5efa64-d403-4a57-8259-c7b7dfb09667",
    "status": 0
}

Returning Search Results

This function returns search results.

Request:    GET /search/result?id=[UUID]&limit=[max records]
Result:     200 with JSON structure SearchResult. Check the field status.
type SearchResult struct {
	Status int                  `json:"status"` // Status: 0 = Success with results, 1 = No more results available, 2 = Search ID not found, 3 = No results yet available keep trying
	Files  []apiBlockRecordFile `json:"files"`  // List of files found
}

Example request: http://127.0.0.1:112/search/result?id=ac5efa64-d403-4a57-8259-c7b7dfb09667&limit=10

Example response with dummy data:

{
    "status": 1,
    "files": [{
        "id": "b5b0706c-817c-492f-8203-5005c59f110c",
        "hash": "Mv6O773ytkJ5jSjLoy2EvHQaM5KfVppJHeTppMc7alA=",
        "type": 1,
        "format": 14,
        "size": 10,
        "folder": "",
        "name": "88d8cc57d5c2a5fea881ceea09503ee4.txt",
        "description": "",
        "date": "2021-09-23T00:00:00Z",
        "metadata": []
    }]
}

The user can terminate a search early using this function. This helps save system resources and should be considered best practice once a search is no longer needed (for example when the user closes the tab or window that shows the results).

Request:    GET /search/terminate?id=[UUID]
Response:   204 Empty

Start Download

This starts the download of a file. The path is the full path on disk to store the file. The hash parameter identifies the file to download. The blockchain parameter is the node ID of the peer who shared the file on its blockchain (i.e., the "owner" of the file).

Request:    GET /download/start?path=[target path on disk]&hash=[file hash to download]&blockchain=[node ID]
Result:     200 with JSON structure apiResponseDownloadStatus
type apiResponseDownloadStatus struct {
	Status int `json:"status"` // Status: 0 = Success, 1 = Error download not found
}

Get Download Status

This returns the status of an active download. The hash and blockchain parameters must be the same as /download/start.

Request:    GET /download/status?hash=[file hash]&blockchain=[node ID]
Result:     200 with JSON structure apiResponseDownloadStatus

Explore

List Recently Shared Files

This returns recently shared files in Peernet. Results are returned in real-time. The file type is an optional filter.

Request:    GET /explore?limit=[max records]&type=[file type]
Result:     200 with JSON structure SearchResult. Check the field status.

Example request to list 20 recently shared files (all file types): http://127.0.0.1:112/explore&limit=20

Example request to list 10 recent documents: http://127.0.0.1:112/explore?type=5&limit=10

Helper Functions

These helper functions are usually not needed, but can be useful in special cases.

Detect file type and file format

This function detects the file type and file format of the specified file. It will primarily use the file extension for detection. If unavailable, it uses the first 512 bytes of the file data to detect the type.

Request:    GET /file/format?path=[file path on disk]
Result:     200 with JSON structure apiResponseFileFormat
type apiResponseFileFormat struct {
	Status     int    `json:"status"`     // Status: 0 = Success, 1 = Error reading file
	FileType   uint16 `json:"filetype"`   // File Type.
	FileFormat uint16 `json:"fileformat"` // File Format.
}

Example request: http://127.0.0.1:112/file/format?path=test.txt

Example response:

{
    "status": 0,
    "filetype": 1,
    "fileformat": 10
}