Author(s): [Adrien Delorme]
Last updated: 2019-10-15
Discussion at https://golang.org/issue/33974.
Move already existing code residing in golang/go/src/cmd/go/internal/lockedfile to x/sync.
A few open source Go projects are implementing file locking mechanisms but they do not seem to be maintained anymore:
lockedfile package API is more ergonomic. Incompatibilities with AIX, Solaris and Illumos are preventing file locking on both projects, but it looks like the go team is addressing for lockedfile.As a result some major projects are doing their own version of it; ex: terraform, boltdb. After some researches it seemed to us that the already existing and maintained lockedfile package is the best ‘open source’ version.
File-locking interacts pretty deeply with the os package and the system call library in x/sys, so it makes sense for (a subset of) the same owners to consider the evolution of those packages together. We think it would benefit the mass to make such a package public: since it's already being part of the go code and therefore being maintained; it should be made public.
We propose to copy the golang/go/src/cmd/go/internal/lockedfile to x/exp. To make it public. Not changing any of the named types for now.
Exported names and comments as can be currently found in 07b4abd:
// Package lockedfile creates and manipulates files whose contents should only
// change atomically.
package lockedfile
// A File is a locked *os.File.
//
// Closing the file releases the lock.
//
// If the program exits while a file is locked, the operating system releases
// the lock but may not do so promptly: callers must ensure that all locked
// files are closed before exiting.
type File struct {
// contains unexported fields
}
// Create is like os.Create, but returns a write-locked file.
// If the file already exists, it is truncated.
func Create(name string) (*File, error)
// Edit creates the named file with mode 0666 (before umask),
// but does not truncate existing contents.
//
// If Edit succeeds, methods on the returned File can be used for I/O.
// The associated file descriptor has mode O_RDWR and the file is write-locked.
func Edit(name string) (*File, error)
// Transform invokes t with the result of reading the named file, with its lock
// still held.
//
// If t returns a nil error, Transform then writes the returned contents back to
// the file, making a best effort to preserve existing contents on error.
//
// t must not modify the slice passed to it.
func Transform(name string, t func([]byte) ([]byte, error)) (err error)
// Open is like os.Open, but returns a read-locked file.
func Open(name string) (*File, error)
// OpenFile is like os.OpenFile, but returns a locked file.
// If flag implies write access (ie: os.O_TRUNC, os.O_WRONLY or os.O_RDWR), the
// file is write-locked; otherwise, it is read-locked.
func OpenFile(name string, flag int, perm os.FileMode) (*File, error)
// Read reads up to len(b) bytes from the File.
// It returns the number of bytes read and any error encountered.
// At end of file, Read returns 0, io.EOF.
//
// File can be read-locked or write-locked.
func (f *File) Read(b []byte) (n int, err error)
// ReadAt reads len(b) bytes from the File starting at byte offset off.
// It returns the number of bytes read and the error, if any.
// ReadAt always returns a non-nil error when n < len(b).
// At end of file, that error is io.EOF.
//
// File can be read-locked or write-locked.
func (f *File) ReadAt(b []byte, off int64) (n int, err error)
// Write writes len(b) bytes to the File.
// It returns the number of bytes written and an error, if any.
// Write returns a non-nil error when n != len(b).
//
// If File is not write-locked Write returns an error.
func (f *File) Write(b []byte) (n int, err error)
// WriteAt writes len(b) bytes to the File starting at byte offset off.
// It returns the number of bytes written and an error, if any.
// WriteAt returns a non-nil error when n != len(b).
//
// If file was opened with the O_APPEND flag, WriteAt returns an error.
//
// If File is not write-locked WriteAt returns an error.
func (f *File) WriteAt(b []byte, off int64) (n int, err error)
// Close unlocks and closes the underlying file.
//
// Close may be called multiple times; all calls after the first will return a
// non-nil error.
func (f *File) Close() error
// A Mutex provides mutual exclusion within and across processes by locking a
// well-known file. Such a file generally guards some other part of the
// filesystem: for example, a Mutex file in a directory might guard access to
// the entire tree rooted in that directory.
//
// Mutex does not implement sync.Locker: unlike a sync.Mutex, a lockedfile.Mutex
// can fail to lock (e.g. if there is a permission error in the filesystem).
//
// Like a sync.Mutex, a Mutex may be included as a field of a larger struct but
// must not be copied after first use. The Path field must be set before first
// use and must not be change thereafter.
type Mutex struct {
// Path to the well-known lock file. Must be non-empty.
//
// Path must not change on a locked mutex.
Path string
// contains filtered or unexported fields
}
// MutexAt returns a new Mutex with Path set to the given non-empty path.
func MutexAt(path string) *Mutex
// Lock attempts to lock the Mutex.
//
// If successful, Lock returns a non-nil unlock function: it is provided as a
// return-value instead of a separate method to remind the caller to check the
// accompanying error. (See https://golang.org/issue/20803.)
func (mu *Mutex) Lock() (unlock func(), err error)
// String returns a string containing the path of the mutex.
func (mu *Mutex) String() string
The lockedfile.File implements a subset of the os.File but with file locking protection.
The lockedfile.Mutex does not implement sync.Locker: unlike a sync.Mutex, a lockedfile.Mutex can fail to lock (e.g. if there is a permission error in the filesystem).
lockedfile adds an Edit and a Transform function; Edit is not currently part of the file package. Edit exists to make it easier to implement locked read-modify-write operation. Transform simplifies the act of reading and then writing to a locked file.
Making this package public will make it more used. A tiny surge of issues might come in the beginning; at the benefits of everyone. (Unless it's bug free !!).
There exists a https://godoc.org/github.com/rogpeppe/go-internal package that exports a lot of internal packages from the go repo. But if go-internal became wildly popular; in order to have a bug fixed or a feature introduced in; a user would still need to open a PR on the go repo; then the author of go-internal would need to update the package.
There are no retro-compatibility issues since this will be a code addition but ideally we don‘t want to maintain two copies of this package going forward, and we probably don’t want to vendor x/exp into the cmd module.
Perhaps that implies that this should go in the x/sys or x/sync repo instead?
Adrien Delorme plans to do copy the exported types in the proposal section from cmd/go/internal/lockedfile to x/sync.
Adrien Delorme plans to change the references to the lockedfile package in cmd.