util.persistence.pRef module¶

exception util.persistence.pRef.DuplicateNameError¶

Bases: Exception

Raised when a name lookup matches more than one PRObject.

exception util.persistence.pRef.MalformedIDError¶

Bases: Exception

Raised a PReference is syntactically incorrect.

exception util.persistence.pRef.CircularReferenceError¶

Bases: Exception

Raised when a PRDb document (directly or via a chain of other documents currently being read) ends up referencing its own URL while that URL’s PRDb._read() is still in progress. Without this check, such a document would otherwise send PRDb.__new__() into unbounded recursion – each nested attempt to load the same not-yet-finished URL passing every earlier guard (it’s not yet in AllDBs; the file exists; it’s not the top-level _reading call) – until the interpreter’s call stack is exhausted.

class util.persistence.pRef.PRObject(db: DBT, **kwargs)¶

Bases: Serializable, Dirtyable, Generic

An object with an immutable integer ID and a mutable name.

__init__(db: DBT, **kwargs) → None¶

Parameters:

db (PRDb) – The db to which this object will belong.
kwargs (Any) – Passed on to super().__init__(). PRObject is often one link in a longer chain of cooperative __init__ calls – eg: a class combining AbstractGraph’s AGRelation with a sibling class via multiple inheritance (see sn.model.model.Relation) reaches PRObject through more than one base. Calling super().__init__(**kwargs) here (instead of simply returning) ensures whatever comes after PRObject in that particular class’s MRO – eg: AttributedObject, for a Relation or plain GraphObject – still gets initialized, exactly once, with the constructor arguments meant for it, even though PRObject itself has no use for them.

Raises:

PermissionError – If the DB is write locked.
TypeError – If the value is not of type PRObject.
MemoryError – if the DB has been deleted.

Note:

This used to call Serializable.__init__(self) and Dirtyable.__init__(self) directly, up here, before the super().__init__(**kwargs) call below. Serializable has no __init__ of its own so that call was a no-op, but the explicit Dirtyable call meant Dirtyable got initialized via a hardcoded direct call rather than via this class’s cooperative super() chain – and, since a class combining PRObject with something listed after Dirtyable in its own MRO (eg: sn.view.graphics.vobjects.VObject, which also mixes in ErrorReportingObject) relies on that same chain to reach the classes after Dirtyable too, this dead end silently meant ErrorReportingObject.__init__() – and anything else listed after Dirtyable – never ran at all for any such object. Removing the explicit calls and letting Dirtyable (now itself cooperative – see its own __init__) be reached exactly once via super() below fixes that for every current and future user of this class, not just this one case.

_db: ReferenceType¶

_id: int¶

classmethod getInstantiationInfo() → InstantiationInfo¶

Returns:: A InstantiationInfo structure describing the constructor parameters and how to fill them (conArgs, conKWArgs), and a further description of any additional data to be filled after immediately after the constructor (postconSetters) and more after any contained objects have been fully constructed (finalSetters), both how to write them and read them.
Return type:: util.persistence.serialize.InstantiationInfo

property id: int¶

Returns:: this object’s ID in its DB.
Return type:: int

property db: DBT¶

Returns:: this object’s ID.
Return type:: :py:class:’PRDb
Raises:: MemoryError – if the DB has been deleted.

abstract property name: str¶

Returns:: this object’s (usually mutable) name.
Return type:: str

Note

This is an abstract method (property) used to resolve persistent references of the for <url>#<string-name>.

getPRef() → PReference[Self]¶

Returns a new PReference object referencing this object.

Returns:
Return type:: PReference[Self]

torepr(relativeTo: DBT) → str¶

Parameters:: relativeTo (DBT)
Returns:
Return type:: str

__str__() → str¶

Returns:
Return type:: str

tostring(relativeTo: DBT | None = None) → str¶

Parameters:: relativeTo (DBT | None)
Returns:
Return type:: str

_dbUrl() → str¶

The URL of our DB, or the one it had when we were created if it has gone.

Returns:
Return type:: str

class util.persistence.pRef.PRContainer(db: DBT, **kwargs)¶

Bases: PRObject, Generic

A PRObject that contains other objects (see getMembers()).

abstractmethod getMembers() → Iterator[PRObject[DBT]]¶

Returns:
Return type:: Iterator[PRObject[DBT]]

class util.persistence.pRef.PRDbContainer(db: DBT, **kwargs)¶

Bases: PRContainer, Generic

The object standing for its whole database (as object 0): its members are all the database’s objects.

getMembers() → Iterator[PRObject[DBT]]¶

Returns:
Return type:: Iterator[PRObject[DBT]]

property name: str¶

Returns:
Return type:: str

class util.persistence.pRef.PRDb(url: str, mode='w', memoryResident: bool = True, _reading: bool = False)¶

Bases: Serializable, Subject, Dirtyable, Generic

A dict-like object whose keys are restricted to int and values to PRObject.

__getitem__() and related lookup methods accept either:

an int -> direct key lookup by ID
a str -> lookup by PRObject.name (raises DuplicateNameError if ambiguous)

METHODS PURPOSELY NOT IMPLEMENTED:

__setitem__() (index assignment) is NOT implemented. Instead, the constructor
of PRObject calls PRDb._register() to obtain an id and insert itself into the PRDb.

update()

AllDBs: ClassVar[dict[str, PRDb]] = {}¶

aliases: ClassVar[dict[str, str]] = {}¶: Other URLs for DBs – eg: their old names, or where they were before they were moved – and the URLs they stand for (see resolveURL()): eg, “sn:/sn.xml” -> “sn:/sn”. One ending in “/” stands for everything under it (see _alias()).

namedURLs: ClassVar[dict[str, Callable[[], str]]] = {}¶: Names – pseudo-URLs, eg: “sn:/baseRules” – of DBs whose actual URLs are only known when the program runs (eg: a temporary copy of a file that comes with it), and how to get each one’s URL – reading the DB, if need be (see resolveURL()). A reference to one is written as its name (see relativeURL()).

_namesOf: ClassVar[dict[str, str]] = {}¶: The names of the namedURLs that have been resolved, by their actual URLs.

readProgress: ClassVar[Callable[[PRDb, int], None] | None] = None¶: If set, called as a file’s objects are read – every READ_PROGRESS_STEP of them – with the DB being read and how many of its objects have been read so far (eg: to show a progress bar: its nextID is about how many there’ll be).

READ_PROGRESS_STEP: ClassVar[int] = 250¶: How many objects are read between calls of readProgress.

_readingURLs: ClassVar[set[str]] = {}¶: URLs for which a PRDb._read() call is currently in progress (see __new__() below). A URL is added just before its _read() starts and removed as soon as that call returns or raises. __new__() checks this set before starting a new read so that a document which (directly, or through a chain of other documents it references) ends up loading its own URL again fails fast with a CircularReferenceError instead of recursing until the stack overflows: AllDBs alone can’t catch this, since a URL is only added there after its read completes.

classmethod _initCache()¶: Re-initializes the cache.

Warning

This should only be used for testing purposes (eg: unit testing) as it wipes the DB cache and could cause bad pointers in any interface applications.

Note

Subclasses should override to do whatever needs to be done for “blank” cache.

classmethod getDB(url: str, mode='w') → Self¶

Parameters:

url (str) – The name of the db as a file name or a URL.

Returns:

The DB read from disk or from the web.

Return type:

PRDb

Raises:

FileNotFoundError – If a file: URL points to a path that does not exist.
ValueError – if the url is malformed.
urllib.error.URLError – If a remote URL cannot be reached.

Either return the DB from memory or attempt to read it from disk or from the web.

Parameters:: mode (str)

classmethod _read(url: str, mode='w') → Self¶

Parameters:

url (str) – The URL to fetch. May use any scheme supported by urllib.request, plus the file: protocol for local filesystem paths.

Returns:

Raw byte content of the resource, suitable for passing directly to an XML parser (e.g. xml.etree.ElementTree._fromstring() or lxml.etree.XMLParser).

Return type:

bytes

Raises:

ValueError – If url is an empty string.
FileNotFoundError – If a file: URL points to a path that does not exist.
urllib.error.URLError – If a remote URL cannot be reached.

Read the DB from disk or from the web and instantiate it. Subclasses should implement this method to actually do the read; this version merely raises the exception.

Parameters:: mode (str)

classmethod readFile(url: str, mode='w') → Any¶

Read the object in the file (or at the URL) url – usually a PRDb, but not necessarily – and return it. Use this rather than SerializationStrategy.readx() to read a PRDb, as it takes care of files that have been copied or moved:

A copied or moved file still has its old URL in it. That’s found out first (without reading the file), so that while it’s read it’s taken as being the file it was actually read from – never confused with the original (which may be open) – and so are its references to files that were next to it, if they’ve been copied along with it (see _relocate()). Its previousURL is the URL it was copied from (written in the header the next time it’s saved).

Parameters:

url (str)
mode (str)

Returns:

Return type:

Any

_wasRead(url: str) → None¶

Called once this DB has been read from url (see readFile()): for subclasses to do whatever’s needed then. (Here, nothing.)

Parameters:: url (str) – Where it was read from.

_bases: list[str] = []¶: The URLs of the files being read or written just now, innermost last: relative URLs in a file are relative to the file (see relativeURL(), absoluteURL()).

classmethod basedOn(url: str | None)¶

Within this context, relative URLs are relative to url (eg: the file being read or written) – unless url is None.

Parameters:: url (str | None)

static isRelativeURL(url: str) → bool¶

Whether url is a relative URL, eg: “pets.snm” or “shared/animals.snm” (not “file:…”, “sn:…”, “/…”, “$name”, “#12”).

Parameters:: url (str)
Returns:
Return type:: bool

classmethod absoluteURL(url: str, base: str | None = None) → str¶

url, if it’s relative, made absolute against base – by default the file being read or written (see basedOn()); otherwise url as is (eg: a path typed by the user, relative to the current directory).

Parameters:

url (str)
base (str | None)

Returns:

Return type:

str

MAX_UP: int = 2¶: The most folders up (“../”) a relative URL goes (see relativeURL()): further than that, URLs stay absolute.

classmethod relativeURL(url: str, base: str | None = None) → str¶

url relative to base – by default the file being read or written (see basedOn()) – if it’s a file in the same folder, below it, or at most MAX_UP folders up from it, eg: “pets.snm”, “shared/animals.snm” or “../animals.snm”; otherwise url as is (eg: a file in an unrelated folder, or anything not a file). So a block of files copied together keeps referring to one another. (A file copied on its own still finds what it referred to: see absoluteURL().)

Parameters:

url (str)
base (str | None)

Returns:

Return type:

str

_readFrames: list[tuple[str, str]] = []¶: (stored URL, actual URL) for each copied or moved file being read just now (see _read()).

_relocated: set[tuple[str, str]] = {}¶: (from, to) relocations already reported.

FILE_SCHEMES: ClassVar[tuple[str, ...]] = ('', 'file', 'http', 'https')¶: The schemes of URLs of files (on this computer or elsewhere). Others – eg: “sn:” – are pseudo-URLs: names of DBs in memory (see namedURLs), taken as they are.

classmethod resolveURL(url: str, mode: str = '') → str¶

Parameters:

url (str) – A DB’s URL, as given: perhaps relative (see absoluteURL()), an alias (see aliases), or a name (see namedURLs).
mode (str) – As for fixURL(): what to check it’s there for (“w”: writing; “”: nothing).

Returns:

The URL it stands for: made absolute, given the default extension if it’s a file’s without one (see fixURL()), and relocated if it’s in a file that’s been copied or moved (see _relocate()). A pseudo-URL (see FILE_SCHEMES) is as it is (but for aliases and names).

Return type:

str

classmethod _alias(url: str) → str¶

Returns:: What url is an alias for (see aliases), if it is; else url. An alias ending in “/” stands for everything under it: eg, with “http://old.org/onto/” -> “https://new.org/ontologies/”, “http://old.org/onto/family.snm” is “https://new.org/ontologies/family.snm”.

classmethod _relocate(url: str) → str¶

While a copied or moved file is being read (see _read()): its old URL is taken to be the one it was read from; and a URL of a file that was next to it (eg: the model of a copied view) is taken to be the same file next to it now – if that file is there. Otherwise, url.

Parameters:: url (str)
Returns:
Return type:: str

What to put in the head of its document when this is written (see SerializationStrategy.tostring()), besides its URL and super-DBs (see getInstantiationInfo()): the URL it was copied or moved from, if it was; and how many objects are in it (see objectCount()).

Returns:
Return type:: dict[str, HeadValue]

objectCount() → int¶

Returns:: How many objects are written when this is (see serializationHeader()): by default, all of them (not its container).
Return type:: int

__init__(url: str, mode: str = 'w', memoryResident: bool = True, _reading: bool = False)¶

Parameters:

url (str) – The URL to associate with this object. Can be a filename, interpreted in the context of the current directory.
_reading (bool) – (default False) Should not be used. Used by the persistence system to get around the existence of the URL during construction.

Raises:

ValueError – If the URL is malformed.
DuplicateNameError – If the DB already exists in memory.
FileExistsError – If the URL already exists.

Creates a new PRDb. If the url exists already, you should be using PRDb.getDB() to retrieve it.

Parameters:

mode (str)
memoryResident (bool)

reading: bool¶: Can be used by contained objects to determine if a read operation is taking place.

superDBs: list[Self]¶: List of other DBs that will be searched for indicies ([]) and get(): what this one builds on, as though their contents were its own (but not changed through it). Saved with it (see getInstantiationInfo()); change it with addSuperDB() and removeSuperDB().

_missingSuperDBs: list[str]¶: The (saved) URLs of super-DBs that couldn’t be read when this was: kept, and saved again, so that they’re used once they’re there again.

url: str¶

_db: NoNoneDict[int, T]¶

nextID: int¶

previousURL: str | None¶: The URL this was copied or moved from, if it was (see _read()); written in the header.

classmethod getInstantiationInfo() → InstantiationInfo¶

Returns:: A InstantiationInfo structure describing the constructor parameters and how to fill them (conArgs, conKWArgs), and a further description of any additional data to be filled after immediately after the constructor (postconSetters) and more after any contained objects have been fully constructed (finalSetters), both how to write them and read them.
Return type:: util.persistence.serialize.InstantiationInfo

allSuperDBs() → list[PRDb]¶

Returns:: This one’s super-DBs, theirs, and so on (each once), nearest first.
Return type:: list[PRDb]

addSuperDB(db: PRDb) → None¶

Make db a super-DB of this one (see superDBs), if it isn’t already.

Parameters:: db (PRDb) – The DB.
Raises:: ValueError – if db is this one, or builds on it (directly or not): a cycle.

removeSuperDB(db: PRDb) → None¶

Stop db being a super-DB of this one (see superDBs), if it is.

Parameters:: db (PRDb) – The DB.

property name¶

Returns:
Return type:

save(saveas: str | None = None, overwrite=False, format: str = 'XML3') → None¶

Parameters:

saveas (Optional[str])
overwrite (bool)
format (str)

writeLocked() → bool¶

Returns:: True iff and the DB is write-locked.
Return type:: bool

_register(value: T, _id: int | None = None) → int¶

Parameters:

value (PRObject) – The PRObject to be added to this DB.
_id (Optional[int]) – Normally (and defaulted to) None. But if value is an object that has previously been unregistered, this value is the the original ID (that had been returned from this method) and will be restored to this ID.

Returns:

A new (locally) unique id for this object.

Raises:

PermissionError – If the DB is write locked.
TypeError – If the value is not of type PRObject.

Generate a (locally) unique id for this object and add it the this DB.

The method is private, but is used in the constructor of PRObject.

_unregister(_id: int) → None¶

Parameters:

value (PRObject) – The PRObject to be added to this DB.
_id (Optional[int]) – Normally (and defaulted to) None. But if value is an object that has previously been unregistered, this value is the the original ID (that had been returned from this method) and will be restored to this ID.

Returns:

A new (locally) unique id for this object.

Raises:

ValueError – if _id does not exist.

Generate a (locally) unique id for this object and add it the this DB.

The method is private, but is used in the constructor of PRObject.

setDirty(dirty: bool = True) → None¶

An object is dirty if it has been modified and should probably be saved on exit. This should be called from all methods that modify the state of this object.

If dirty is False, resets all the contained Attributes to False.

Parameters:: dirty (bool) – (default True) The dirty to set dirty to.

isDirty() → bool¶

Returns True iff the object (or an of it’s contained Attributes) have been modified and should probably be saved on exit.

Returns:
Return type:: bool

findByName(name: str) → list[T]¶

Parameters:: name (str)
Returns:
Return type:: list[T]

_resolve(key: int | str) → int¶

Return the integer key for key, without necessarily checking for it’s existence.

int -> returned as-is (existence check deferred to caller)

str -> scanned for a matching .name; DuplicateNameError if >1 match

Parameters:

key (int|str) – The index or name of the object.

Returns:

the int index of the object.

Return type:

int

Raises:

DuplicateNameError – if a str index is not unique.
KeyError – if the str index is not found.
TypeError – if the key is not either an int of a str.

__getitem__(key: int | str) → T¶

Parameters:: key (int | str)
Returns:
Return type:: T

_makeDBContainer() → PRDbContainer[Self]¶

Returns:
Return type:: PRDbContainer[Self]

getall(key: int | str, filtr: Callable[[T], bool] = <function PRDb.<lambda>>, throw: bool = False) → list[T]¶

Return the integer key for key.

int -> returned as-is (existence check deferred to caller)
str -> scanned for a matching .name; DuplicateNameError if >1 match

Parameters:

key (int | str)
filtr (Callable[[T], bool])
throw (bool)

Returns:

Return type:

list[T]

get(key: int | str, default: T | None = None) → T | None¶

Parameters:

key (int | str)
default (T | None)

Returns:

Return type:

T | None

keys() → KeysView[int]¶

Returns:
Return type:: KeysView[int]

values() → ValuesView[T]¶

Returns:
Return type:: ValuesView[T]

items() → ItemsView[int, T]¶

Returns:
Return type:: ItemsView[int, T]

torepr(relativeTo: Self | None = None) → str¶

Parameters:: relativeTo (Self | None)
Returns:
Return type:: str

__str__() → str¶

Returns:
Return type:: str

tostring(relativeTo: Self | None = None) → str¶

Parameters:: relativeTo (Self | None)
Returns:
Return type:: str

class util.persistence.pRef.PReference(text: str)¶

Bases: Generic

A reference to a persistent object implementing the __call__() method for resolving the referernce into a pointer.

__init__(text: str)¶

Parameters:

text (str) – A string in the form of <URL>#<id>, where <id> can be an int, a string with no spaces or a quoted string.

Raises:

MalformedIDError – if the form of the ID is not valid.
ValueError – If it isn’t a valid URL.
FileNotFoundError – If the file or URL does not exist.

Creates a reference to a PRObject without actually trying to instantiate the PRObject or checking on it’s existence. To instantiate the object, use a call on this object.

url: str | None¶

name: str | None¶

id: int¶

obj: T | PRDb | None¶

__call__(relativeTo: PRDb[T] | str | None = None) → T | PRDb[T]¶

Returns:: the resolved PRObject.
Raises:: LookupError – if the reference cannot be resolved.
Parameters:: relativeTo (PRDb[T] | str | None)

tostring(relativeTo: PRDb[T] | None = None) → str¶

Parameters:: relativeTo (PRDb[T] | None)
Returns:
Return type:: str

__str__() → str¶

Returns:
Return type:: str

torepr(relativeTo: PRDb[T] | None = None) → str¶

Parameters:: relativeTo (PRDb[T] | None)
Returns:
Return type:: str

classmethod formatForURL(ref: str | Self | T1, contextDB: PRDb | None = None, includeName=True) → str¶

Parameters:

ref (str|PReference|PRObject) – A string or PReference or PRObject.
contextDB (PRDb) – A db to generate a relative reference string to. To shorten the string. This with a “x?id=10: will be “http:///x?id=10” with a context and merely “?id=10” with context=x

Returns:

The canonical for an ID. Since URL-fragment parts are not transmitted to the server in requests, they cannot be used to request fragment parts of an object, so where “#<id>” might be expected to be used, “?id=<id>” is used instead.

Format:

<url>?{id=<id_int>}{{&}name=<name>}

Parameters:: includeName (bool)

classmethod eval(text: str, contextDB: ~util.persistence.pRef.PRDb | None = None, filtr: ~typing.Callable[[T1], bool] = <function PReference.<lambda>>) → T1 | PRDb¶

Evaluate the text expression as a PReference to an object. But be friendly in interpreting the expression by assuming a # prefix is missing if we can’t find the db in memory, file system, or web.

Parameters:

text (str) – A valid string representing a PReference.
contextDB (Optional[PRDb]) – A PRDb in which to evaluate text.
filtr (Callable[[PRObject],bool]|None) – An optional PRObject->bool function to use to disambiguate between multiple possible return values if necessary. It should return True for a possible candidate.

Returns:

A PRObject represented by text.

Return type:

PRObject

classmethod _isValidID_old(text: str) → tuple[str | None, int | str]¶

Parameters:

text (str) – The text of a PReference in the form <URL>#[<int>|<str>].

Returns:

A 2-tuble [url, id] where url is a valid URL or None, and id is either an int or a string

Raises:

MalformedIDError – if the form of the ID is not valid.
ValueError – If it isn’t a valid URL.
FileNotFoundError – If the file or URL does not exist.

Splits the id into a URL part and an ID part. Validates the existence of the URL but makes no attempt to validate the ID. The ID may be either an int or a string name.

<validID> := <absoluteURL> | <relativeURL>

<absoluteURL> := <existing-URL>#<id>

<relativeURL> := {#}<id>

<id> := int | <str-without-whatespace> | <quoted-str>

<validID> := <absoluteURL> | <relativeURL>

<absoluteURL> := <existing-URL>{?<parameters>}#<id>

<parameters> := name=<name> | id=<int>

<relativeURL> := {#}<id>

<id> := <int> | <str-without-whatespace> | <quoted-str>

classmethod _isValidID(s: str) → tuple[str | None, int | None, str | None]¶

Split a string into path, id, and name components.

Respects single and double quoted substrings and backslash escapes. The ‘#’ and ‘?’ delimiters may appear in any order, but are expected to appear at most once each outside of quoted regions.

Query string must be of the form ‘name=<str>’ or ‘id=<int>’, where <str> may be a quoted string or an HTML percent-encoded string.

Fragment is a convenience for gracious interpretation of user input only. It must be an integer, a quoted string, or an HTML percent-encoded string. And if it’s a an integer, will be interpreted as an id; if it’s an string, will be interpreted as name. If it disagrees with the query field, a ValueError exception is thrown.

Locations are specified by standard http: https: or file: URLs, however the special notation “$<name>” looks up a DB in memory by it’s name alone as a user interface convenience.

Parameters:

s (str) – The string to split, e.g. a URL or path with query/fragment.

Returns:

A 3-tuple of (path, name, id). name is a str or None, id is an int or None.

Return type:

tuple[str, str | None, int | None]

Raises:

ValueError –

If the query or fragment cannot be parsed or one of the query fields doesn’t match the fragment field.

util.persistence.pRef.db: PRDb¶

util.persistence.pRef.a: CPObject¶

util.persistence.pRef.b: CPObject¶

util.persistence.pRef.c: CPObject¶