Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 27 additions & 17 deletions man/shareData.Rd
Original file line number Diff line number Diff line change
Expand Up @@ -4,43 +4,53 @@
\alias{clearData}
\title{ Share Data between R Sessions}
\description{
Experimental functions that enable the user to share a R object between 2 \R sessions.
Share one \R object between two \R sessions on the same machine.
}
\usage{
shareData(data, map_name, verbose=FALSE)
getData(map_name, verbose=FALSE)
clearData(x, verbose=FALSE)
}
\arguments{
\item{data}{ A \R object like a vector or a \code{data.frame}.}
\item{map_name}{ A character. A name for the memory map location where to store the data. Needs to start with a "/" on POSIX platforms, this is done automatically if missing.}
\item{x}{ An external pointer like the one returned by function \code{shareData}.}
\item{verbose}{ A logical value \code{TRUE} or \code{FALSE} to provide or not information to the user.}
\item{data}{ An \R object, for example a vector or a \code{data.frame}.}
\item{map_name}{ A character string. The name of the shared memory object. The function adds a leading \code{/} if it is missing. Use a simple name. Do not use a path or a Windows \code{Global} or \code{Local} prefix. A name can describe only one object: calling \code{shareData()} again under a name that is in use overwrites that object (or fails if the new object is larger), and the older pointer's \code{clearData()} then leaves the newer object in place.}
\item{x}{ An external pointer returned by \code{shareData}.}
\item{verbose}{ A logical value. Use \code{TRUE} to print progress information, or \code{FALSE} to suppress it.}
}
\details{
\code{shareData} is a deep copy, not a pointer: the object is serialized into a raw vector in the sharing session and that raw vector is copied into a shared memory object. \code{getData} copies the raw vector back into the reading session and unserializes it. The full object is thus serialized and unserialized once and copied twice (into and out of shared memory), which for large objects dominates the cost of the transfer.
\strong{This is a copy, not a live pointer.}
\code{shareData()} serializes \code{data} to a raw vector and copies it into shared memory. \code{getData()} copies the raw vector back and unserializes it. The object is copied twice. Large objects thus take time and memory. A plain pointer into the R heap of another session cannot work, because R objects carry process-private metadata (garbage collection, string cache, ALTREP, ...). Issue #43 (\url{https://github.com/fastverse/kit/issues/43}) tracks the request for a lightweight pointer-based alternative.

The shared object can be read once: \code{getData} unlinks the memory map as soon as it has read the data, so calling \code{getData} again with the same \code{map_name} fails.
\strong{Lifetime and platform differences.}
\describe{
\item{POSIX (Linux, macOS, and other Unix-like systems)}{
\code{getData()} unlinks the shared memory object before it calls \code{unserialize()}. A second call with the same \code{map_name} thus fails. The object stays alive while the pointer returned by \code{shareData()} exists. \code{clearData()} unmaps and unlinks the object it created, unless another session has since reused the name; then it only cleans up its own view. The same cleanup occurs when the pointer is garbage collected or the session exits. If the sharing session is killed, the object can stay in \code{/dev/shm} on Linux. It stays there until you remove it or the machine restarts.
}
\item{Windows}{
\code{getData()} closes its handles but does not delete the mapping. A second call thus succeeds while the sharing session is alive. \code{clearData()} removes the mapping when the last handle is closed. If the sharing session is killed, Windows closes its handles and the mapping disappears.
}
}

\strong{Scope.}
Sharing works only on one machine and for the same user. It does not send data over a network. Use the same \R version and the same version of \pkg{kit} when possible. External pointers and connections do not survive. Environments are copied, not shared.

The data lives as long as the external pointer returned by \code{shareData} exists in the sharing session: \code{clearData} unmaps and unlinks the shared memory object and returns \code{TRUE}, or \code{FALSE} if it was already cleared. The same cleanup happens automatically when that pointer is garbage collected or the session exits. If the sharing session is killed before any of that, the shared memory object may persist (\code{/dev/shm} on POSIX) until removed.
\strong{Memory limits.}
The operating system limits shared memory. On Linux, \code{/dev/shm} is often smaller than RAM, especially in containers. A large object can thus fail even when the machine has enough memory.

Functions are currently only tested on POSIX platforms (using \code{shm_open}) and Windows (using named file mappings), other platforms are untested.
The functions are currently tested on POSIX platforms (with \code{shm_open}) and Windows (with named file mappings). Other platforms are untested.
}
\value{
\code{shareData} returns a external pointer.
\code{getData} returns an \R object stored in the memory location \code{map_name}.
\code{clearData} returns \code{TRUE} or \code{FALSE} depending on whether the data have been cleared in memory.
}
\note{
Sharing objects is not zero-copy: a plain pointer into the R heap of another session cannot work, because R objects carry process-private metadata (garbage collection, string cache, ALTREP, ...). Issue #43 (\url{https://github.com/fastverse/kit/issues/43}) tracks the request for a lightweight pointer-based alternative.
\code{shareData} returns an external pointer.
\code{getData} returns the \R object stored at \code{map_name}.
\code{clearData} returns \code{TRUE} if it released the pointer's mapping, or \code{FALSE} if the pointer was already cleared.
}
\author{Morgan Jacob}
\examples{
\dontrun{
# In R session 1: share data in memory
x = shareData(mtcars,"share1")
x = shareData(mtcars, "share1")

# In R session 2: get data from session 1 (readable only once!)
# In R session 2: get data from session 1
getData("share1")

# In R session 1: clear data in memory
Expand Down
Loading