CRCTable is the default table used for crc64 sum calculations
var CRCTable = crc64.MakeTable(crc64.ISO)
ErrDuplicatePath occurs when a tar archive has more than one entry for the same file path
var ErrDuplicatePath = errors.New("duplicates of file paths not supported")
Entries is for sorting by Position
type Entries []Entry
func (e Entries) Len() int
func (e Entries) Less(i, j int) bool
func (e Entries) Swap(i, j int)
Entry is the structure for packing and unpacking the information read from the Tar archive.
FileType Payload checksum is using `hash/crc64` for basic file integrity, _not_ for cryptography. From http://www.backplane.com/matt/crc64.html, CRC32 has almost 40,000 collisions in a sample of 18.2 million, CRC64 had none.
type Entry struct {
Type Type `json:"type"`
Name string `json:"name,omitempty"`
NameRaw []byte `json:"name_raw,omitempty"`
Size int64 `json:"size,omitempty"`
Payload []byte `json:"payload"` // SegmentType stores payload here; FileType stores crc64 checksum here;
Position int `json:"position"`
}
func (e *Entry) GetName() string
GetName returns the string for the entry's name, regardless of the field stored in
func (e *Entry) GetNameBytes() []byte
GetNameBytes returns the bytes for the entry's name, regardless of the field stored in
func (e *Entry) SetName(name string)
SetName will check name for valid UTF-8 string, and set the appropriate field. See https://github.com/vbatts/tar-split/issues/17
func (e *Entry) SetNameBytes(name []byte)
SetNameBytes will check name for valid UTF-8 string, and set the appropriate field
FileGetPutter is the interface that groups both Getting and Putting file payloads.
type FileGetPutter interface {
FileGetter
FilePutter
}
func NewBufferFileGetPutter() FileGetPutter
NewBufferFileGetPutter is a simple in-memory FileGetPutter
Implication is this is memory intensive... Probably best for testing or light weight cases.
FileGetter is the interface for getting a stream of a file payload, addressed by name/filename. Presumably, the names will be scoped to relative file paths.
type FileGetter interface {
// Get returns a stream for the provided file path
Get(filename string) (output io.ReadCloser, err error)
}
func NewPathFileGetter(relpath string) FileGetter
NewPathFileGetter returns a FileGetter that is for files relative to path relpath.
FilePutter is the interface for storing a stream of a file payload, addressed by name/filename.
type FilePutter interface {
// Put returns the size of the stream received, and the crc64 checksum for
// the provided stream
Put(filename string, input io.Reader) (size int64, checksum []byte, err error)
}
func NewDiscardFilePutter() FilePutter
NewDiscardFilePutter is a bit bucket FilePutter
Packer describes the methods to pack Entries to a storage destination
type Packer interface {
// AddEntry packs the Entry and returns its position
AddEntry(e Entry) (int, error)
}
func NewJSONPacker(w io.Writer) Packer
NewJSONPacker provides a Packer that writes each Entry (SegmentType and FileType) as a json document.
The Entries are delimited by new line.
Type of Entry
type Type int
const (
// FileType represents a file payload from the tar stream.
//
// This will be used to map to relative paths on disk. Only Size > 0 will get
// read into a resulting output stream (due to hardlinks).
FileType Type = 1 + iota
// SegmentType represents a raw bytes segment from the archive stream. These raw
// byte segments consist of the raw headers and various padding.
//
// Its payload is to be marshalled base64 encoded.
SegmentType
)
Unpacker describes the methods to read Entries from a source
type Unpacker interface {
// Next returns the next Entry being unpacked, or error, until io.EOF
Next() (*Entry, error)
}
func NewJSONUnpacker(r io.Reader) Unpacker
NewJSONUnpacker provides an Unpacker that reads Entries (SegmentType and FileType) as a json document.
Each Entry read are expected to be delimited by new line.