Bifroest Mr. Tines MacCTC CTCjava Manual Pages
Bifroest Mr. Tines CTC Home CTClib MacCTC CTCjava Manual

Portable I/O module

This is also known as the virtual files module (hence the VF prefix on routines).

Introduction

CTC requires a file handling module, primarily to handle the multiple passes over the data required by PGP and clones. This may be extremely efficiency critical on machines with poorly optimised file systems. Ultimately the aim is to reduce the number of physical passes through the data by maximising the Virtual files provides the following facilities:- Whereas these facilities could be highly machine dependent it is intended that they should also be implementable using the C <stdio.h> library, with the anticipation that they will be efficient if the underlying library is efficient.

File Types

The Virtual File system supports the following types of file:-

Disc File

These are named permanent files out in the machines filing system. They may be declared as read-only, write-only or read-write. They are used as the final input or final output. The standard Virtual Filing system does not include primitives for opening, closing, creating or deleting these. Such operations are machine and filing system dependent. However, there are example operations which should be implemented by machines using the C <stdio.h> interface. The Virtual File system does include an operation to wipe files by filling them with pseudorandom data .

Buffer

This is a temporary file. The Virtual Files implementation may store some, all or none of the information in a buffer on disc. The data in the file will ultimately be discarded. A buffer may be declared sensitive if it contains data that should not be revealed. The storage of a sensitive file should wipe before it is release. For efficiency, buffers that have not been declared sensitive should not be wiped. A buffer may either be read-write or write-then-read. If of the later type it is write only until the first read operation then it is read only. A compound is a file made up of one or more parts of one or more buffers and disc files. Compound files are read-only. Once a buffer or disc file has been incorporated into a compound file it also becomes read-only . VF References Each Virtual File regardless of type is referenced via a DataFile object. This object is a structure with contents known only to the VF module. As such calling modules do not even know the size of this structure, all references are in terms of pointers to this type normally referenced as the pointer type DataFileP. Each Virtual File has a reference count indicating the number of times that the file is point. This should represent the number of valid DataFilePs in existance for the VF. If this falls to zero, the file is closed. When a Disc file closes, the VF module calls the O/S specific vf_disc_close routine which must be provided to close the disc file. If a buffer is closed, it is wiped if declared sensitive, and then all its storage is released. If a compound file is closed, the reference count of each component is decremented, and they may be themselves closed as a consequence. The definitions of all VF module routines must specify the effect of the routine on the reference count. If a VF is passed to a routine there may be one of three semantics.
Borrowed Ref. [B]
The subroutine uses the DataFileP reference for the duration of the call only and responsibility for closing the file remains with the calling program.
Copied Ref. [C]
The subroutine may increment the reference count for the file and keep a copy of the reference.
Assumed Ref [A]
The subroutine may taken over responsiblity for the reference. The calling program may not use the reference again.

Access Modes

There are three access modes. There are:-
Ascii Mode
In ASCII mode, the calling program reads or writes a line at a time. The read operations return a null terminated string with no trailing carriage control. The write operations accept similar strings. The carriage control characters (if any) on a write operation are stripped and replaced by carriage control characters specific to the file.
Caller Buffer Mode
This is a binary access mode. The calling program passes the VF module a buffer to be written to the file or read from it. This is the same as ASCII mode but without any carriage control conversion.
VF Buffer Mode
This is also a binary access mode. However in this mode the buffer is provided by the VF module. The calling program requests a buffer from the VF module. If it is a read mode the buffer is filled with data from the file. Once the access to the buffer is complete, the calling program releases the buffer, and if it is a write mode the data is written to the file. No other VF operations may be carried out on the file (or its components) while such a buffer is in use. (The main purpose of this mode is to allow small files to implemented in-memory without copy operations.)

DataTypes

DataFile
This is a private structure that contains the details of an open file. Objects of this type cannot be declared by calling programs as its size is private.
DataFileP
This is a pointer to DataFile and is the type declared in calling programs to manipulate files.
CCmode
Carriage Control mode: This defines the why in which new-line characters are handled when writing text lines to the file. Currently this always consists of appending a specific control-character or characters to the end of the line. However other formats, like prefixing line length, would also possible if required. The following values are defined:-
CANONICAL
This is the PGP canonical form. Text files are converted to this form for both encryption or signing. In practice this is the same as PC
LOCAL
This is the form of carriage control used on the machine that the program is built to run on. Whereas this is typically the same as one of the other formats, it could be radically different from any of them. There is not even any guarantee that end-of-line is indicated by imbedded control characters. Other mechanisms (e.g. prefixed line-length) are also possible.
PC
As per MS-DOS, i.e. <CR><LF>
UNIX
As per Unix, i.e. <LF>
MAC
As per MacOS, i.e. <CR>
accessMode
This defines the mode in which a file will be accessed. Note that this more complex than most filing systems will implement. Implementations of this module may but need not enforce access to comply appropriate access mode. The module must make best endeavours to destroy all data on closure if the file is in a "wipe" mode. However note that the operation of many operating system facilities (e.g. Virtual Memory) makes it impractical for many implementations to guarantee this. (This data-type is actually defined in abstract.h.)
READ
Read only
READWIPE
Read only then wipe
WRITE
Write only
UPDATE
Read/write (may mix)
UPDATEWIPE
Read/write then wipe on close
WRITEREAD
Write then read: the file becomes read-only on the first read.
WRITEREADWIPE
Write then read then wipe
fileType
This defines the purpose of a file. (This data-type is actually defined in abstract.h.)
TEXTCYPHER
Text (armoured) cyphertext
BINCYPHER
Binary cyphertext
TEXTPLAIN
Text plaintextfileType
BINPLAIN
Binary plaintext
PUBLICRING
Public keyring
SECRETRING
Secret keyring
vf_error
ILLEGAL
Illegal operation attempted
UNIMPLEMENTED
Requested operation is not implemented in the version.
ENDOFFILE
End of file encountered
OSERROR
Operating system reported an error
WRONGMO
too many of something, typically files open
NO_MEMORY
memory allocation refused

Functions

Whereas these routines will be the normal method of opening and closing files in many environments they need not be the only method nor necessarily available at all in some environments

Optional Entry Points

These entry points are optional. Their specifications are provided to provide consistency across applications that do handle files in this way.
DataFileP vf_open(char * name, accessMode mode, fileType type);
This opens a file by name. This routine is not practical to provide with some filing systems and accordingly should NOT be called by CTClib code.
byte * vf_buffer(long * size, DataFileP[B] file);
This function and vf_release() were in the original specification, but have not in fact been used. They provide direct access to a file by locking it in memory.

 

 

On entry *size is the requested size of the buffer. This is ignore if the file mode is read-only. On exit *size is the size of the buffer returned. If the file mode is write-only this should unchanged. If the file mode is read-only it should be the file size. If the file mode is read/write it should be the larger of the two. The file image is locked and the address of the start of the buffer is returned. The function returns NULL if it fails.

Note if the file mode is read-only, it is a program error to update the buffer. For the duration that the file is accessible in this way, it is not allowed to access it in any other way.

long vf_release(long size, DataFileP[B] file);
This function releases a buffer acquired by vf_buffer(). If the file mode is writable, the size of the file is set to size and the file is updated to consist of the first size bytes of the buffer. (N.B. size must be less than or equal to the *size value returned by vf_buffer().)

File Operations

DataFileP vf_toReplace(DataFileP[B] old);
This routine and vf_replaceWith are intended to be called as a pair. They are used to where a new file is being written to replace an existing file, and it is necessary to refer to the unmodified original file in the course of writing the new file. (This is necessary in CTC for rewriting public key-rings.)

 

 

Creates a new potentially permanent but as yet unnamed (or temporary named) file, in the same area (same partition/directory) as old. The fileType of the result is inherited from old.

A file created with vf_toReplace() should be deleted if it is closed with vf_close(), and will continue to exist with the old file's name if it is closed as second argument to vf_replaceWith() can also be applied to files that have been opened vf_open().

void vf_replaceWith(DataFileP[A] oldFile, DataFileP[A] newFile);
This function closes both files. It renames 'old' to a backup name (.bak or similar), renames 'new' to the former name of 'old' and closes both. Although not necessarily in that order. vf_replaceWith is not guaranteed to work unless oldfile was previous used to call vf_toReplace() and that call returned newfile.
void vf_close(DataFileP[A] file);
Closes the file reference and releases any associated resources. The file reference must not be used again and attempting to do so may cause serious errors.
DataFileP vf_tempfile(long expected_size);
The expected_size argument is a hint to the vf_tempfile routine. It does not matter if it is wrong however ideally it should be a value that is unlikely to be exceeded, but not greatly in excess of the requirement. An expected_size of zero indicates that the caller has no idea how large the file will grow.
DataFileP vf_staticfile(byte * buffer, long length);
This function creates a temporary read-only file with contains defined by buffer and length.
DataFileP vf_copyRef(DataFileP[C] file);
This copies a reference to an open file. Each copy will have its own independant file position.

 

 
 
 

Text Operations

These operations treat the file as a text file. However note that these operations may (and sometimes are) mixed with binary file operations, at least where the CCmode is LOCAL. Accordingly in a number of environments it may be necessary to treat the file as a pseudo-binary file. This is especially important for the generation of file signatures, where the appropriate binary image for the text must be generated.

It is recognised that on any machine that stores text as something other than lines directly concatenated separately by one or two control characters this may be a problem. [This is probably true of some types of IBM data-sets.] On such machines, there may have to be quiet radical differences in the manipulation of a file depending on its Carriage-Control format. In general CANONICAL must be the pseudo-binary, and LOCAL should be the native text format. How UNIX, PC & MAC are handled in these circumstances is left to the judgement of the implementor.

long vf_readline(char * buffer, long buffer_size, DataFileP[B] file);
This reads the next line from the file, returning the line as a NULL terminated string in buffer. Where the line is longer than the buffer, the first buffer_size-1 bytes from the line. The file pointer is then positioned to the first byte in the line that was not returned. The next call will return a "line" starting with that character. Some THINK_C-specific code remains in the sample implementation of this module, unmaintained, as an example of how to handle line-based I/O with a byte-stream style of input.
long vf_writeline(char * buffer, DataFileP[B] file);
This function writes the contents of the buffer followed by the appropriate carriage control characters to the file
long vf_writeline_xt(char * buffer, DataFileP[B] file);
This function is very similar to vf_writeline(). The only difference is that this version does not write any carriage control characters to the file. In practice this behaviour is only important when writing CANONICAL mode files for signature. The PGP signature format does not have a terminator on the final line. Accordingly this routine is used, instead of vf_writeline(), to write the finalline in such cases. At present this is the only use made of this routine. Many implementations will be able to implement this as:-
long    vf_writeline_xt(char * buffer, DataFileP file)
{       return vf_write(buffer, strlen(buffer), file);}
void vf_CCmode(CCmode mode, DataFileP[B] file);
This function sets the Carriage control mode of file to mode.

Binary Operations

long vf_read(void * buffer, long buffer_size, DataFileP[B] file);
Reads buffer_size bytes from the file and puts them in buffer. It returns the number of bytes read. This should be equal to buffer_size unless an error occurred or end-of-file was encountered.
long vf_write(void * buffer, long buffer_size, DataFileP[B] file);
Writes buffer_size from buffer to the file starting from the current position. It returns the number of bytes written. This should be equal to buffer_size unless an error has occurred.

Common Operations

boolean vf_movepos(DataFileP[B] file, long offset);
Moves the cur