Peernet-Archive/core
1
0
Fork 0
mirror of https://github.com/PeernetOfficial/core.git synced 2026-07-17 02:47:51 +01:00
Code Issues Packages Projects Releases Wiki Activity

webapi: New function /blockchain/file/update

Browse Source 
blockchain: Add readme.md. Add function ReplaceFiles.
This commit is contained in:
Kleissner
2021-10-13 20:09:09 +02:00
parent 336de68801
commit b556acb86c
6 changed files with 134 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
blockchain/Blockchain File.go
View File

@@ -2,6 +2,8 @@
File Name: Blockchain File.go
Copyright: 2021 Peernet s.r.o.
Author: Peter Kleissner
Note that files include virtual folders as well for any operation.
*/
package blockchain
@@ -83,3 +85,18 @@ func (blockchain *Blockchain) DeleteFiles(IDs []uuid.UUID) (newHeight, newVersio
return
}
// ReplaceFiles is a convenience wrapper to replace files in the blockchain identified via their IDs. Status is BlockchainStatusX.
// If a file does not exist on the blockchain, it acts as add.
func (blockchain *Blockchain) ReplaceFiles(files []BlockRecordFile) (newHeight, newVersion uint64, status int) {
var deleteIDs []uuid.UUID
for n := range files {
deleteIDs = append(deleteIDs, files[n].ID)
}
if newHeight, newVersion, _, status = blockchain.DeleteFiles(deleteIDs); status != BlockchainStatusOK {
return
}
return blockchain.AddFiles(files)
}

5
blockchain/Blockchain.go
View File

@@ -226,6 +226,7 @@ func (blockchain *Blockchain) IterateDeleteRecord(callback func(record *BlockRec
}
// If refactor, re-calculate the block. All later blocks need to be re-encoded due to change of previous block hash. The version number needs to change.
// Note: Deleting records may leave referenced records orphaned, such as RecordTypeTagData for deleted file records.
if refactorBlock {
if len(newRecordsRaw) > 0 {
blockchainNew = append(blockchainNew, Block{OwnerPublicKey: blockchain.publicKey, RecordsRaw: newRecordsRaw, BlockchainVersion: refactorVersion, Number: uint64(len(blockchainNew))})
@@ -280,6 +281,10 @@ func (blockchain *Blockchain) Append(RecordsRaw []BlockRecordRaw) (newHeight, ne
blockchain.Lock()
defer blockchain.Unlock()
if len(RecordsRaw) == 0 {
return blockchain.height, blockchain.version, BlockchainStatusOK
}
block := &Block{OwnerPublicKey: blockchain.publicKey, RecordsRaw: RecordsRaw}
// set the last block hash first

61
blockchain/readme.md Normal file
View File

@@ -0,0 +1,61 @@
# Blockchain
The blockchain stores the metadata of files published by the user, profile data, and social interactions. The blockchain is implemented according to the Peernet Whitepaper published at [peernet.org](https://peernet.org).
The blockchain is a consecutive sequence of blocks linked together by their previous hash. Each block may contain one or multiple records.
All blocks and the blockchain header are stored locally in a key-value database.
# Encoding
## Header
The blockchain header is not part of the Peernet specification. Below is the encoding of the blockchain header. The public key can be extracted from the signature.
```
Offset Size Info
0 8 Height of the blockchain
8 8 Version of the blockchain
16 2 Format of the blockchain. This provides backward compatibility.
18 65 Signature
```
## Block
Encoding of a block (it is the same stored in the database and shared in a message):
```
Offset Size Info
0 65 Signature of entire block
65 32 Hash (blake3) of last block. 0 for first one.
97 8 Blockchain version number
105 4 Block number
109 4 Size of entire block including this header
113 2 Count of records that follow
```
Each record inside the block has this basic structure:
```
Offset Size Info
0 1 Record type
1 8 Date created. This remains the same in case of block refactoring.
9 4 Size of data
13 ? Data (encoding depends on record type)
```
# Internals
## Block Size
The block size is currently recommended to be slightly below 64 KB (minus message header overhead), so that it fits within a single UDP packet. Having a block size smaller than the max. message size reduces complexity when exchanging individual blocks and increases performance for operations such as file search.
## Edge Cases
### Deleting vs Replacing Records
If a specific record shall be replaced, it should be deleted and a new block containing the replacement record shall be created.
Inline replacement of a record in a block would lead to problems:
* The block size could increase which could push the block size above the recommended limit.
* In case of `RecordTypeFile` records, they may use `RecordTypeTagData` records for compression. If a single record is to be replaced 1:1 with another record, this could not take advantage of this embedded compression algorithm.

1
webapi/API.go
View File

@@ -48,6 +48,7 @@ func Start(ListenAddresses []string, UseSSL bool, CertificateFile, CertificateKe
Router.HandleFunc("/blockchain/self/add/file", apiBlockchainSelfAddFile).Methods("POST")
Router.HandleFunc("/blockchain/self/list/file", apiBlockchainSelfListFile).Methods("GET")
Router.HandleFunc("/blockchain/self/delete/file", apiBlockchainSelfDeleteFile).Methods("POST")
Router.HandleFunc("/blockchain/file/update", apiBlockchainFileUpdate).Methods("POST")
Router.HandleFunc("/profile/list", apiProfileList).Methods("GET")
Router.HandleFunc("/profile/read", apiProfileRead).Methods("GET")
Router.HandleFunc("/profile/write", apiProfileWrite).Methods("POST")

35
webapi/File.go
View File

@@ -215,6 +215,41 @@ func apiBlockchainSelfDeleteFile(w http.ResponseWriter, r *http.Request) {
EncodeJSON(w, r, apiBlockchainBlockStatus{Status: status, Height: newHeight, Version: newVersion})
}
/*
apiBlockchainSelfUpdateFile updates files that are already published on the blockchain.
Request: POST /blockchain/file/update with JSON structure apiBlockAddFiles
Response: 200 with JSON structure apiBlockchainBlockStatus
400 if invalid input
*/
func apiBlockchainFileUpdate(w http.ResponseWriter, r *http.Request) {
var input apiBlockAddFiles
if err := DecodeJSON(w, r, &input); err != nil {
return
}
var filesAdd []blockchain.BlockRecordFile
for _, file := range input.Files {
// Verify that the file exists in the warehouse. Folders are exempt from this check as they are only virtual.
if !file.IsVirtualFolder() {
if hashA, err := warehouse.ValidateHash(file.Hash); err != nil {
http.Error(w, "", http.StatusBadRequest)
return
} else if _, _, valid := core.UserWarehouse.FileExists(hashA); !valid {
EncodeJSON(w, r, apiBlockchainBlockStatus{Status: blockchain.BlockchainStatusNotInWarehouse})
return
}
}
filesAdd = append(filesAdd, blockRecordFileFromAPI(file))
}
newHeight, newVersion, status := core.UserBlockchain.ReplaceFiles(filesAdd)
EncodeJSON(w, r, apiBlockchainBlockStatus{Status: status, Height: newHeight, Version: newVersion})
}
// ---- metadata functions ----
// GetMetadata returns the specified metadata or nil if not available.

15
webapi/readme.md
View File

@@ -36,6 +36,7 @@ These are the functions provided by the API:
/blockchain/self/add/file Add file to the blockchain
/blockchain/self/list/file List all files stored on the blockchain
/blockchain/self/delete/file Delete files from the blockchain
/blockchain/file/update Updates files on the blockchain
/profile/list List all profile fields
/profile/read Read a profile field
@@ -240,6 +241,8 @@ Each file must be already stored in the Warehouse (virtual folders are exempt).
If the block record encoding fails for any file, this function aborts with the status code StatusCorruptBlockRecord. In case the function aborts, the blockchain remains unchanged.
Do not add the same file with the same ID multiple times. Doing so will create double entries. This function does not check if the file is already stored on the blockchain. Storing multiple files with the same file hash, but different IDs, is perfectly fine.
```
Request: POST /blockchain/self/add/file with JSON structure apiBlockAddFiles
Response: 200 with JSON structure apiBlockchainBlockStatus
@@ -372,6 +375,18 @@ Example response indicating success:
}
```
### Update File
This updates files that are already published on the blockchain. This is useful for example when changing a file name or description.
Just like with the add file function, the file must be already stored in the Warehouse, otherwise this function fails.
Note as this replaces the previous file record on the blockchain, all details (including special metadata fields) must be included.
```
Request: POST /blockchain/file/update with JSON structure apiBlockAddFiles
Response: 200 with JSON structure apiBlockchainBlockStatus
```
## Profile Functions
User profile data such as the username, email address, and picture are stored on the blockchain. Profile fields are text (UTF-8) or binary encoded, depending on the type.