Upgrading to 0.6¶
0.6 is a breaking release: it changes the database file format, a number of APIs and some query results. This page lists what to do, grouped by where it affects you: the files, the results of queries, the Rust API, each binding and the command-line tool. The CHANGELOG has every entry, with the issue behind it. What changed from 0.6.0-beta.1 to 0.6.0 is listed here too, for those upgrading from the beta.
Before You Upgrade¶
- Pin the new minor version:
grafeo = "0.6"in Cargo,"@grafeo-db/js": "^0.6.0"in npm,grafeo~=0.6.0for pip and uv (see Versioning). - Stop every 0.5.x process that uses a database before 0.6 opens it. 0.5.x cannot open a migrated database.
- Plan for the migration: the first read-write open of a 0.5.x database migrates it to the 0.6 format and keeps the old files under
.pre-0.6names, which are your way back. The migration needs free disk space for a copy of the database, and write access to the directory that holds it. Read-only opens read a 0.5.x database without changing it, and so do the CLI commands that only read (grafeo info,stats,schema,validate, a query that does not write). See Upgrading from 0.5 for the details, encrypted databases and builds without thewalfeature. - Open every 0.5.x database for writing once with 0.6: 0.7 will no longer read 0.5.x databases.
- Take a full backup after the upgrade, before the next incremental one: incremental backups now copy WAL segments (backup file version 2). A restore still reads the incremental backups of 0.5.x and 0.6.0-beta.1.
- Check the data 0.5.x may have written wrongly (see Data Written by 0.5.x): left-pointing relationships created by Cypher or MERGE, names written in backticks through
SET, and databases compacted by 0.5.40 to 0.5.44.
Database Files¶
- Every database is a single file, whatever its extension, with its write-ahead log in
<path>.wal/while it is open. A 0.5.x WAL directory is migrated to a file at the same path, with its property, text and vector indexes. 0.5.x logged no distance or HNSW parameters for a vector index: the migrated one uses those ofCREATE VECTOR INDEXwithout options (the cosine distance) and takes its dimensions from the vectors, and the open logs a warning that names it. Create a vector index that used other parameters again after the migration. - The write-ahead log has a new format: a WAL written by 0.5.x or 0.6.0-beta.1 that still holds commits is replayed on the first read-write open, checkpointed into the file and removed.
- RDF triples are stored as term ids (from 0.6.0-beta.1): the first read-write open of a file that 0.6.0-beta.1 wrote with RDF triples keeps a copy of the file, and of its WAL, as
<path>.pre-0.6.0, then rewrites its RDF and Ring index sections in the new version at once. A read-only open reads the beta file as it is. 0.6.0-beta.1 cannot open the rewritten file: the.pre-0.6.0copy is the way back. Terms that 0.6.0-beta.1 stored keep the form it stored them in: it dropped a datatype it did not know, stored decimals as doubles and wrote numbers in canonical form. - A database file keeps the configuration it was created with: an open applies the stored graph model, backward edges, schema constraints, WAL and sync mode, memory limit, tier overrides and an in-directory spill path to the settings it does not set itself. An open that sets a different graph model or backward-edge setting fails. To change what a file stores, use
GrafeoDB::set_stored_config. See Stored Configuration. - Opening a directory that is not a 0.5.x database fails, an empty one included. Use a path where nothing exists yet, such as a file inside that directory. Creating a database next to a non-empty
<file>.wal/left from another database fails and names the directory: remove it first. save()fails if the path exists: save to a new path, or remove the old file first.save()andbackup_full()write to<target>.partialand rename it once complete, so you may see that file while they run.- A damaged write-ahead log stops the open with an error that names the file and offset; 0.5.x skipped the damage and went on. To open such a database anyway, replay the commits before the damage with
Config::with_wal_salvage()(Pythonwal_salvage=True, Node.jswalSalvage); the rest of the log is kept in<path>.wal.damaged. - Backups: manifests and cursors are JSON. 0.6 restores 0.5.x backups and continues their chains, but 0.5.x cannot read a backup directory that 0.6 has written to. Once a backup has been taken,
close()keeps the WAL written since the last backup, so a<path>.walcan remain after a clean close.backup_full()no longer checkpoints the database first. - Snapshots are version 5:
export_snapshot()keeps the whole catalog (constraint and index names, index options, graph types, schemas, procedures). Version 4 snapshots from 0.5.x and 0.6.0-beta.1 still import, but earlier releases cannot import a version 5 snapshot. A snapshot does not carry the configuration a database file stores. - A build without a feature refuses the data only that feature reads: a build without
triple-store,vector-indexortext-indexfails to open a database or snapshot holding RDF triples, vector or text indexes, where 0.5.x dropped them silently. Thegrafeocrate's defaults, thelpgandrdfprofiles and the CLI lack some of these: addtriple-store(orrdf) andaito the features if your data has them. - Encryption at rest is enforced: an encrypted database with a spill path, with
TierOverride::ForceDiskor without a path now fails to open, where 0.5.x wrote the data unencrypted. The.pre-0.6copy of a migrated database stays unencrypted. In Rust, the encryption types come with theencryptionfeature of thegrafeocrate, andrestore_to_epochrefuses an encrypted backup with incremental segments: userestore_to_epoch_withand the key chain. - Vector and text indexes are built from the data when a database opens, so an open with large indexes takes longer.
- Ids and epochs continue across reopens: a node or edge id is never reused after a reopen, and epochs continue after a reopen and after the migration, so they differ from what 0.5.x gave. Caches keyed on them may notice.
open_in_memory()leaves the source untouched: no lock and no checkpoint (it opened the file read-write).- Read-only databases refuse every write: SPARQL updates,
restore_snapshot, imports andbatch_insert_rdfnow fail on them. - A write after
close()fails withGRAFEO-T007(PythonDatabaseClosedError), where it was accepted and lost. Reopen the database instead. - With the default features,
GrafeoDB::openreaches the file only at a checkpoint: add thewalfeature for commits that survive a crash.
Data Written by 0.5.x¶
- Properties stored as null are gone after the migration: 0.6 has no null-valued properties, so
keys()andproperties()no longer list them,GCounterandOnCountervalues that 0.5.x wrote as null included. - Left-pointing relationships created by Cypher
CREATEorMERGE, or GQLMERGE, ((a)<-[:T]-(b)) were stored fromatob, backwards. Find and reverse them; 0.6 stores them frombtoa. - Keys and labels written through a backticked name in
SETorREMOVE(SET n.`my key` = 1) were stored with the backticks in the name. Rename them. - Databases compacted by 0.5.40 to 0.5.44 lost data in the compacted store: indexes created before
compact()were dropped (create them again), nodes without labels were dropped, missing properties were stored as'',0,0.0orfalse, lists, maps and temporal values as text, and deleted items come back. See Compact Store.
Query Results¶
These changes apply in every binding.
Numbers¶
- A number with a point is an exact decimal in GQL and SQL/PGQ:
0.1 + 0.2 = 0.3holds, and1.50keeps its two digits. Write1.5e0or1.5Ffor a float, and2Mfor a decimal integer. A decimal with an integer gives a decimal, with a float a float; division andavggive at least 16 significant digits. A decimal assigned to aFLOAT64property is stored as the nearest float. Cypher1.5stays a float. Write1.0e0 / 0.0e0for an infinity. Bindings return decimals asdecimal.Decimal(Python),{"$decimal": "1.50"}(Node.js, WebAssembly, C) and a decimal type of their own (C#, Dart, Go); the Arrow export usesDecimal128. - SPARQL numbers follow the XPath type promotion:
1.5andxsd:decimalliterals are decimals, an integer divided by an integer is a decimal (5 / 2is2.5, it was2), and integer arithmetic past the 64-bit range goes on as decimals. A stored numeric literal computes and compares as a number of its type, and the XPath casts (xsd:decimal(...),xsd:double, ...) convert (they gave no value). - Overflow and division by zero fail the query with
GRAFEO-Q006(GQLSTATUS22003or22012) in GQL, Cypher and SQL/PGQ; they returned null. Guard a division with a condition (WHERE x <> 0 AND 10 / x > 1) if zero can occur.absof the smallest integer fails the same way. A float literal past the float range (RETURN 1e400) is an error too:22003in GQL, a syntax error in Cypher. - An integer past 64 bits is an exact decimal wherever a value comes in: parameters and properties in every binding,
LOAD DATA ... FORMAT JSONLandgrafeo data loadread it as a decimal with scale 0 up to 38 digits (it was a float that lost its last digits, a string, or an error, depending on the path); past 38 digits it is an error naming the parameter, property or line. - Float
=and<>are exact (IEEE 754):0.1e0 + 0.2e0 = 0.3e0(Cypher0.1 + 0.2 = 0.3) is false, and9007199254740993no longer equals9007199254740992.0e0.<,<=,>and>=compare an integer with a float exactly, andNaN <= 1is false. Use decimals for exact fractions. CASTtrims the spaces around a text in GQL:CAST(' 42 ' AS INT64)is 42 (it was null); CyphertoInteger(' 42 ')stays null.CASTconverts booleans and numbers into each other:CAST(true AS FLOAT64)is 1.0 andCAST(0 AS BOOLEAN)is false (both were null).SUMandAVGtake numbers or durations: strings, booleans, lists and maps, or numbers mixed with durations, are a type error22G03(sum(['5', '7'])was 12). Durations now sum and average.- A unary sign reads numbers and durations only:
+'Alix'and-trueare a type error22G03in GQL, Cypher and SQL/PGQ, where+xpassed any value through and-xof a non-number was null. - Grouping and
DISTINCTtake3and3.0as one value, and a float key no longer comes back as an integer.min()andmax()over values of different types no longer depend on the input order (min(['19', '3'])is'19').
Dates, Times and Time Zones¶
SESSION SET TIME ZONEtakes a UTC offset only:'+01:00','-05:30','Z'or'UTC'. A named zone ('Europe/Amsterdam') or any other text fails with22009, where it was stored and never used. The time zone now carries over to the next one-shotexecute.- A zoned datetime or time compared with a local one: GQL and SQL/PGQ read the local value at the session time zone (UTC without one, so nothing changes unless you set a time zone). Cypher keeps zoned and local values apart, as openCypher does:
datetime('2024-03-15T10:00:00+01:00') = localdatetime('2024-03-15T09:00:00')is false and<between them is null, where both read the local value at UTC. - Cypher
datetime()andtime()are zoned and keep their offset:hourofdatetime('2024-03-15T10:00+01:00')is 10 (it was 9), and stored values are zoned. Uselocaldatetime()andlocaltime()for local values. GQLzoned_datetimeandzonedTimewithout an offset are at+00:00(they were null). toStringof a local datetime has no zone:2024-03-15T10:00:00, not2024-03-15T10:00:00.000000Z.- A duration times or divided by a number keeps its value:
P1D / 2isPT12H(it wasP0D); division by zero is22012and overflow22003.date,timeanddatetimeof a zoned value, and arithmetic on zoned values, give values where they gave null.
Patterns, Writes and Paths¶
- A
WHEREorFILTERright afterINSERT,CREATE,MERGEorDELETEis an error in GQL: put the condition before the write (MATCH (n) WHERE n.age > 30 DETACH DELETE n). - A statement that ends with a write returns no rows and no columns: a GQL
INSERTreturned the node it created (in_anon_0) and a Cypher write returned internal columns. The write counters are unchanged. - Cypher reads arrows, types and predicates as openCypher does:
<-->and<-[:T]->match either direction,CREATEof a relationship without a type is an error (it got the typeRELATED), an undirectedCREATEis an error, and1 = 2 IN [false]is false. - MERGE matches the whole pattern or creates all of it, and no longer reuses an unconnected node. Cypher
SET n = {...}inON CREATEorON MATCHreplaces the properties. AMERGEof an untyped, alternative-typed or variable-length relationship is an error. - Cypher matches a relationship at most once per
MATCH, and an undirected pattern takes a self-loop once. - Paths:
DIFFERENT EDGES,TRAIL,ACYCLICandSIMPLEapply to the whole path;ANYkeeps one path per pair andSHORTEST kkeeps k paths; a path variable holds every edge. The 100-hop cap is gone: an unbounded variable-length pattern fails past a memory budget instead (as do SPARQL+and*paths). A shortest orANYpattern over several edges is an error. - Nodes, edges and paths in lists and maps are nodes, edges and paths:
RETURN [n, 1],{k: n}andRETURN pgave ids, andstartNode(r)andendNode(r)returned node ids. A result'snodes()andedges()include the nested ones. - A list or map literal keeps missing values:
[p.w, p.age]is[null, 35]and{w: p.w}is{w: null}, sosize()counts them.size()of a string counts characters, not UTF-8 bytes. - Column names: an unaliased
RETURN length(p)is namedlength(p)(it was_path_length_p), and an unaliased SQL/PGQ item is named after its text (count(*)). CREATE GRAPH dst AS COPY OF srccopies the graph (it created an empty one).USE GRAPH gfollowed by a statement in one string runs that statement on g, andSELECT ... FROM g MATCH ...reads graph g; both did nothing or read the current graph.- Snapshot isolation covers values, labels and deletes: other sessions and direct reads no longer see an open transaction's changes to them.
Functions, Types and New Errors¶
- An unknown function, or a call with a number of arguments the function does not take, is an error (#570), where it returned null: among them the
*OrNulland*Listconversions (toStringOrNull),isEmpty,randomUUID,point,distance,btrim,mod,concat,substr,startsWith,endsWith,round(x, 2)andvector($v, 384).EXPLAINof such a query fails too.path_length(p)now works. - These statements now fail, most of them returned null or wrong rows: an aggregate in
WHERE, an aggregate with the wrong number of arguments, an aggregate item next to a non-key item, a SQL/PGQSELECTof a non-key or an aggregate inCOLUMNS,DISTINCTwithORDER BYa dropped variable,DELETEof an unbound variable, an invalid=~pattern,-[:A:B]->, andCREATEwith:A|B. IS TYPEDtakes every type it can check:IS TYPED NODE,EDGEandANYare true, a node is no longerINT64, and an ISO type Grafeo does not support is a42000error.- Type DDL refuses property types Grafeo does not support (
INT32, a typo such asSTIRNG), naming the supported ones; they declared a property of typeANY. It also refuses a default its property cannot hold. - Graph types are enforced: closed graph types and schema-scoped types check writes, graph-type defaults apply, and
DROP GRAPH TYPEof a type in use is refused. Writes that used to pass now fail withG2000; a type created without a schema no longer checks writes in schemas1. AFLOAT64property takes integers (stored as floats). - Vectors: a dimension mismatch, a NaN component or
dimensions=0is aGRAFEO-V001error. - A statement nests at most 64 levels deep and its plan at most 128; a deeper statement fails before it runs. Property values nest at most 128 levels deep, and the migration names any value that is deeper.
- A stored procedure's arguments are values: an undeclared parameter is an error, a float arrives as a float, and a wrong-typed or row-dependent argument in
CALLis an error (parameters and expressions were ignored). A procedure call after other clauses runs once per row.CALL grafeo.dfsreports the tree depth asdepth(it was the finishing order), with newdiscoveryandfinishcolumns: four columns withoutYIELD. - Other functions:
normalizegives NFC,LIKEmatches across newlines, cosine distance is exact,id()andproperties()take any expression, andHOME_SCHEMA,HOME_GRAPHandSESSION RESETgo to the session's home.
Aggregates and Ordering¶
- GQL sorts nulls last in both directions;
ORDER BY x DESCput them first. WriteNULLS FIRSTto keep the old order. Cypher keeps openCypher's order. - Grouped
collectleaves nulls out, and groupedmin,maxandsamplereturn values where they returned null. Cypherlistaggandgroup_concattake their separator (the default is a comma; it was a space). Two-argument aggregates,DISTINCTin the statistical aggregates and GQLHAVINGwithGROUP BYgive corrected results. - SQL/PGQ aggregates inside
SELECTandHAVINGexpressions aggregate (0.5.44 gave one row per match with nulls),GRAPH_TABLE (g MATCH ...)reads graph g (it read the current graph; a missing graph is42002), property maps after the first node andLIMITafterGROUP BYapply, and-[:T*]->means one or more hops. - Deterministic order:
changes_betweenreturns changes in commit order, search results with equal scores in id order, and change data capture events carry their commit timestamps.
Errors, Codes and Conflicts¶
- GQLSTATUS names the kind of error: semantic errors other than an unresolved name move from
42002to42000; a negativeLIMITorSKIPparameter is22G02and a non-integer one22G03; Cypher DDL in a read-only transaction is25G03. A schema or constraint violation isG2000and aDELETEof a node with edgesG1001(both were22000). - Codes: an unsupported feature is
GRAFEO-Q004, a timeout alwaysGRAFEO-Q003, and a damaged fileGRAFEO-S002. A user mistake has a code of its own;GRAFEO-X001now only means a bug in Grafeo. Every query error message starts with its code (GRAFEO-Q002: semantic error: ...): update code that matches messages. - More writes conflict, with the retryable
GRAFEO-T001: commits are pipelined, so a write to something an in-progress commit wrote fails; a UNIQUE constraint holds between concurrent transactions (the second writer getsGRAFEO-T001, orG2000once the value is committed); a direct write while a transaction is open is a transaction of its own and fails on what that transaction changed first;DROP GRAPHrefuses a graph an open transaction changed; and SPARQLCLEAR,DROP,COPY,MOVEandADDwait for open changes in their graphs. Retry these. - A commit whose WAL write fails is not reported as committed:
GRAFEO-T009can be retried, andGRAFEO-T010means the outcome is unknown: reopen the database. After a commit that stopped midway, commits, checkpoints and saves fail withGRAFEO-T008until the database is reopened; reads still work. create_vector_indexin a build without thevector-indexfeature returns an error.
SPARQL and Gremlin¶
- SPARQL results hold RDF terms: each IRI, blank node and literal of a SELECT, CONSTRUCT or DESCRIBE row is the exact term, with its lexical form, language tag and datatype (
"30"^^xsd:integerwas the integer 30,"Pomme"@frthe stringPomme). See Bindings for the type each binding gives. Read a literal's native value where you need a number (Literal.to_python()in Python), and pass terms back as parameters unchanged. - SPARQL operators and ORDER BY follow SPARQL 1.1: a string and a number are never equal (
"42" = 42is false,"42" < 43drops the FILTER row), two plain strings compare as strings ("9" < "10"is false), a tagged literal equals only a literal with its tag,STRgives the lexical form as written ("019"), and ORDER BY sorts numbers by value with unbound values first. Write a typed literal (42,"42"^^xsd:integer) where you meant a number. - SPARQL:
ROUND(-2.5)is -2, a double divided by 0 is INF,isNumeric("1")is false, language tags survive expressions andDATATYPEof a tagged literal isrdf:langString,AVGof integers is anxsd:decimal,SUMandAVGof nothing are 0,ASKreturns one row with the columnboolean,COUNTin a transaction counts its own writes, andDROP ALLworks. - Gremlin filters inside
and(),or(),not()andwhere()apply, and a step Grafeo does not support is an error.
Algorithms and Query Metadata¶
- Graph algorithms return their rows in node-id order, in
CALLand indb.algorithms. Corrected results: k-core (core numbers), Louvain (merges level by level), strongly connected components (numbered in another order), label propagation communities and max-flow flows,as_networkx()PageRank on an undirected graph, clustering coefficients and triangle counts ignore self-loops, k-truss uses the simple graph, andcreate_projectionreads the selected graph. Some betweenness scores differ in their last bits. Do not depend on component or community ids. rows_scannedcounts the rows a query reads from storage: it repeated the rows returned.- Every query reports
execution_time_ms: Cypher, SQL/PGQ, Gremlin, GraphQL, SPARQL,EXPLAIN,SHOWand session commands reported none. PROFILEandEXPLAINshow the operators that ran: an operator a fast path absorbed has no line of its own, and a variable-length expand prints[*1..2].- A statement without a result has GQLSTATUS
00001(omitted result) inQueryResult::gql_status: a write withoutRETURN,FINISH, a schema or transaction command. It was00000. - The Arrow export keeps lists, maps and durations typed (Python
to_arrow()and the polars and pandas conversions, Node.jstoArrowIpc(), the CLI's Arrow dump): they were text. A column of integers and floats is floats; other mixed columns stay text. - File imports keep every column and key: a CSV without headers names its columns
column0,column1, ... (it imported nodes without properties), and headers with spaces or punctuation are kept. - A property whose name starts with
_(_created) is kept on result nodes and edges in every binding, where a name like a structural key replaced the node's identity or was dropped.
Rust¶
- The minimum Rust version is 1.99.0.
- Build
Configwith its constructors and methods: it is#[non_exhaustive], so a struct literal no longer compiles. Its public fields are deprecated: read a setting with its getter. A field written directly does not count as set, so a database file's stored value wins over it on open.
// 0.5
let config = Config { path: Some("data.grafeo".into()), wal_enabled: false, ..Config::default() };
// 0.6
let config = Config::persistent("data.grafeo").without_wal();
- Option and result types are
#[non_exhaustive]: buildEncryptionConfigwithEncryptionConfig::new,CdcRetentionConfigwithCdcRetentionConfig::unlimited()and itswith_*methods,EdgeUpsertOptionswithEdgeUpsertOptions::new(), andDurabilityMode::batch(..)orDurabilityMode::adaptive(..). Match a result type (DatabaseInfo,DatabaseStats,WalStatus,IndexInfo, the schema infos,ValidationResult,ChangeEvent,IndexDefinition, the backup manifest, segment and cursor,CommitInfo,MetricsSnapshot,WriteCounters,UpsertSummary) or aBatchorAdaptivemode with.., and buildMemoryUsageand its parts fromDefault::default().admin::DumpMetadatais removed. - More calls return
Result:drop_graph,create_property_index,drop_property_index,drop_vector_index,drop_text_index,batch_insert_rdf,create_projection, the index calls onSession,Session::set_time_zone, andget_node_at_epoch,get_edge_at_epochandget_node_property_at_epoch(Ok(None)when it did not exist then, an error for an epoch older than the history kept). Add?. A failedrebuild_vector_indexorrebuild_text_indexkeeps the old index. - A read of an earlier epoch reads the database's history:
execute_at_epoch,execute_as_of,Session::set_viewing_epochand the*_at_epochreads fail for an epoch outside the retention window. Without one, a read of an epoch before the last write fails, where a build withouttemporalanswered it with the values of now. SetConfig::with_history_retention(HistoryRetention::Duration(..))orForeverto keep history. text_searchandhybrid_searchtake afiltersargument: passNoneto search as before.compact()returns aCompactReport(admin::CompactionStatsis replaced). The compacted store's API is gone:layered_store(),compact_tiered()andgrafeo::database::compact_tiered, ascompact()no longer builds a separate store.- Errors: a damaged file is
Error::Corruption(Corruption { what, file, offset };StorageError::Corruptionis removed),TransactionStatehas aCommittingstate an exhaustive match needs an arm for, andTerm::from_ntriplesreturnsResult<Term, TermParseError>(RdfStore::load_ntriplesrefuses unknown escapes). - Removed:
Config::wal_flush_interval_mswithConfigError::ZeroWalFlushInterval(set the flush withwith_wal_durability),GrafeoDB::adaptive_config(), the snapshot and section methods ofGrafeoDB::file_manager()(it returns the new file manager), and the deprecatedTieredStoretrait (useSectionandMemoryConsumer).wal()returns the new WAL writer,Option<&Arc<grafeo_storage::wal::Wal>>.
grafeo::Error Is 32 Bytes¶
Its large payloads are boxed. ?, to_string(), error_code(), the GQLSTATUS and the messages do not change; code that builds or matches these variants with their payloads does:
| Before | Now |
|---|---|
Error::Query(query_error) | Error::from(query_error) (or ?); a match binds a Box<QueryError>, whose fields read as before |
Error::Storage(storage_error) | Error::from(storage_error); match Error::Storage(e), then *e or &**e |
Error::TypeMismatch { expected, found } | Error::type_mismatch(expected, found); match Error::TypeMismatch(mismatch), then mismatch.expected and mismatch.found |
Error::Constraint { kind, message: String } | message is a Box<str> (message.into()) |
A Read the Store Cannot Do Is an Error¶
The direct reads of GrafeoDB (get_node, get_edge, get_node_labels, get_node_edges, the history reads, find_nodes_by_property), its counts (node_count, edge_count, label_count, property_key_count, edge_type_count) and reports (info, detailed_stats, schema, validate, AdminService), the direct reads of Session (get_node, get_edge, get_node_property, get_node_edges, the neighbor reads, node_exists, edge_exists, get_degree, get_nodes_batch, find_nodes_by_property, the history reads) and AsOfView::get_node, get_edge and get_node_property return a Result, around the Option where absence is an answer. A direct read of a selected graph that was dropped fails with "Graph '...' does not exist", and a store without history answers the history reads with the unsupported error. For the bindings, see Bindings.
| Before | Now |
|---|---|
db.get_node(id) → Option<Node> (also get_edge, get_node_labels) | db.get_node(id)? → Option<Node> |
db.node_count() → usize (also edge_count, label_count, property_key_count, edge_type_count) | db.node_count()? |
db.get_node_edges(..), db.find_nodes_by_property(..), db.get_node_history(id) → Vec<_> | the same call with ? |
db.info(), db.detailed_stats(), db.schema(), db.validate() | the same call with ? |
session.node_exists(id) → bool, session.get_degree(id) → (usize, usize) | the same call with ? |
Reading a Graph: Snapshot¶
GrafeoDB::snapshot() (the graph set_current_graph selects), GraphHandle::snapshot(), AsOfView::snapshot() and GrafeoDB::projection(name) give a Snapshot: the graph's store, the ReadView its reads take, and a pin that keeps what that view reads while it lives, so several reads see one state across later commits.
| Before | Now |
|---|---|
db.graph_store(), db.selected_graph_store()? | db.snapshot()?, then snapshot.store() and snapshot.view() |
db.graph("g")?.graph_store()? | db.graph("g")?.snapshot()? |
db.projection("p") → Option<Arc<dyn GraphStore>> | db.projection("p") → Option<Snapshot> (snapshot.shared() for the store as a shared handle) |
GrafeoDB::store() stays, marked experimental, for admin work. It returns Result<Arc<RowGroupStore>> (grafeo_core::graph::rowgroup), the store the database now holds its graphs in: LpgStore is gone, with LpgStoreSection, PropertyStorage, NodeRecord, EdgeRecord and LpgStore::copy_graph, and QueryProcessor::for_lpg, Planner::with_lpg_store and ProcedureContext::lpg_store take the row-group store. Write through sessions or the direct API: a write through the store's own methods is not logged and not undone by a rollback.
Reading Through the Store Contract¶
GraphStore has one family of reads, each taking the ReadView that names who reads (ReadView::committed(epoch), or a transaction's view) and returning a Result; GraphStoreSearch is merged into it, and the reads without a view, the _versioned and _at_epoch families and answers_at_view are gone. Calling these reads is stable; implementing the trait is experimental through 0.6.x (see API Stability).
| Before | Now |
|---|---|
get_node(id), get_node_versioned(id, epoch, tx), get_node_at_epoch(id, epoch) (also edges) | node(id, view)? |
get_node_property(id, &key) (also edges, _batch forms) | node_property(id, key, view)?, node_property_batch(ids, &key, view)? |
edges_from(node, dir), edges_from_at(..), edges_of_type(..) | edges(node, dir, &types, view)? |
out_degree(node), in_degree(node) | degree(node, dir, view)? |
node_ids(), all_node_ids(), filter_visible_node_ids_at(..) | node_ids(view)?, filter_visible_node_ids(ids, view)? |
nodes_by_label(label), nodes_by_label_count_at_epoch(..) | nodes_by_label(label, view)?, nodes_by_label_count(label, view)? |
node_count(), edge_count() | node_count(view)?, edge_count(view)? |
all_labels(), all_edge_types(), all_property_keys() → Vec<String> | the same call with ? |
find_nodes_by_property(..), find_nodes_in_range(..), text_search(..), vector_search(..) | the same call with a view last, and ? |
estimate_label_cardinality(..), estimate_avg_degree(..) | statistics() |
current_epoch() | committed_epoch() |
get_node_history(id) | node_history(id)? |
A store of your own implements node, edge, edges, has_backward_adjacency, node_ids, all_labels, all_edge_types, all_property_keys, statistics and committed_epoch; every other read has an exact default. The graph algorithms (grafeo_adapters::plugins::algorithms) take the store and a view and return a Result, and GraphAlgorithm::execute takes the view too.
A Store of Your Own: GraphBackend¶
GrafeoDB::with_store takes an Arc<dyn GraphBackend>, the store contract's write side, whose transactions the engine opens, commits and rolls back. The database gives such a store rollbacks, savepoints and the rollback of a failed statement; what only the built-in store has (named graphs, indexes, search, the database file, imports, backups) is the unsupported error on it. With Config::persistent(path) it is durable: it writes <path>.wal and takes one process at a time, and GraphBackend::durable_epoch and flush say what it holds across a restart. grafeo::store holds the contract's types, and the conformance feature the suite a store runs. See Storage Backends.
| Before | Now |
|---|---|
GrafeoDB::with_store(Arc<dyn GraphStoreMut>, config) | GrafeoDB::with_store(Arc<dyn GraphBackend>, config) |
impl GraphStoreMut for MyStore (create_node, set_node_property, the _versioned writes) | impl GraphBackend for MyStore: reserve_node_ids, reserve_edge_ids, begin, apply(op, writer), commit, rollback, rollback_to, collect, durable_epoch, flush |
ChangeTarget::stamp(transaction, entries, epoch) | GraphBackend::commit(handle, changes, epoch) |
ChangeTarget::undo(transaction, entries) | GraphBackend::rollback(handle, changes), or rollback_to(handle, savepoint, changes) |
store.create_node(..) through GraphStoreMut on a RowGroupStore | the same call, a method of RowGroupStore (drop the trait import) |
GraphWriter::new(store, view), .with_recording(recording) | GraphWriter::new(store, recording); GraphWriter::immediate(store) for writes that commit at once |
Recording { target: WriteTarget::Store(store), .. } | Recording { target: store, .. } (an Arc<dyn GraphBackend>) |
db.graph_store_mut() | none: write through a session or the direct API |
QueryProcessor::lpg_store() → &Arc<RowGroupStore> | Option<&Arc<RowGroupStore>> (None on a backend of one's own) |
db.store() → Arc<RowGroupStore> (a panic on a backend of one's own) | db.store()? → Result<Arc<RowGroupStore>>: the unsupported error there |
db.iter_nodes(), db.iter_edges() → an iterator | db.iter_nodes()?, db.iter_edges()?: the unsupported error on a backend of one's own |
GraphStoreMut, ExternalTarget, UndoSupport and WriteTarget are gone. A QueryProcessor outside a transaction runs a write in a transaction of its own, and set_current_graph refuses a graph such a database does not have.
Bindings¶
Every binding:
- A read the store cannot do is an error (see above): Python raises
GrafeoError, Node.js and WebAssembly throw with the error'sgqlStatus. - An integer past 64 bits is an exact decimal (see Numbers).
- SPARQL results hold RDF terms (see SPARQL and Gremlin): Python gives
grafeo.IRI,grafeo.Literal(with.language,.datatypeand.to_python()) andgrafeo.BlankNode, allstrsubclasses; Node.js and WebAssembly give RDF/JS term objects ({ termType: 'Literal', value: '30', language: '', datatype: { termType: 'NamedNode', value: 'http://www.w3.org/2001/XMLSchema#integer' } }); C#, Go and Dart give anRdfTerm; the C library's JSON gives each term in a$termtag around its SPARQL 1.1 JSON form ({"$term": {"type": "uri", "value": "..."}}). Each result says what it holds (kind: rows, a boolean for ASK, or a graph for CONSTRUCT and DESCRIBE). - Writes after
close()fail (GRAFEO-T007). - Direct single writes are slower than in 0.5.44 (about 0.1 to 0.2 microseconds more per call): write many nodes or edges with the batch calls (
batch_create_nodes_with_props,batch_create_edges) or the upserts, which commit once.
Per binding:
- Python:
db.algorithms.kcore()returns{"core_numbers": {...}, "max_core": n}, likelouvain(), andcompact()returns a dict.get_node_at_epochand the other*_at_epochreads raiseGrafeoErrorfor an epoch older than the history kept (they returnedNone).- A damaged file raises
GrafeoCorruptionError(aGrafeoError); the vector, text and hybrid searches raiseGrafeoError(still aRuntimeError); a write afterclose()raisesDatabaseClosedError. bytesandbytearraycome back asbytes(they were a list of integers), a time with an offset comes back as an awaredatetime.time, anddatetime.timeanddatetime.datecan be stored and passed as parameters.- Long-running calls release the GIL: use a transaction from one thread at a time, since a
commit()orrollback()while another thread runs one of its queries raisesRuntimeError. as_networkx(),as_solvor(),get_nodes_by_label,get_property_batchanddb.algorithmsread the graphset_graph()selects (they read the default graph), and the store reads list only committed data.import_dfimports all rows or none.nodes_df()andedges_df()give the same values with pyarrow installed as without it.grafeo.features()no longer liststemporal.- A SPARQL result's
grafeo.IRI,grafeo.Literalandgrafeo.BlankNodecompare and hash as RDF terms (kind, text, language tag, datatype), as in rdflib: a term no longer equals the plain string of its text. Compare with a term (row["s"] == grafeo.IRI("http://ex.org/a")) or withstr(term). Language tags are lowercase and compare case-insensitively. - Node.js:
compact()returns an object.- The package's entry module is
grafeo.js(types ingrafeo.d.ts): change deep imports of the old entry file. Errors carrygqlStatusand are typed asGrafeoError. - Parameters, batch and upsert rows, edge properties and search filters convert from the JavaScript value as properties do: a whole number past 2^32 is an
INT64(it was a float), aBigIntof any width follows the integer rule, and the strings"NaN"and"Infinity"stay strings (they became floats). Errors name the parameter, or the row and property. ResultStream.next()returnsPromise<Record<string, unknown> | null>(it wasPromise<JsonValue | null>): update type annotations.- WebAssembly:
compact()returns an object, andimportRowsandimportLpgimport all rows or none.- Parameters, rows and filters convert from the JavaScript value as in Node.js, and a
Dateparameter is a timestamp. - C:
grafeo_node_countandgrafeo_edge_countreturn a status and write the count to a newsize_t*argument.grafeo_get_node_labelsandgrafeo_as_of_get_node_propertyreturn a status and write the JSON to a newchar**argument (NULL when the node or property does not exist).grafeo_get_node,grafeo_get_edge,grafeo_graph_get_node,grafeo_graph_get_edge,grafeo_as_of_get_nodeandgrafeo_as_of_get_edgereturnGRAFEO_OKwith NULL for a node or edge that does not exist (they returnedGRAFEO_ERROR_DATABASE): check the pointer afterGRAFEO_OK.grafeo_inforeturns NULL with the error.grafeo_drop_property_index,grafeo_drop_vector_index,grafeo_create_projectionandgrafeo_drop_projectionreturn -1 on an error; the two projection calls return anint32_t.- Invalid params or properties JSON is an error with its reason, where the call ran without them.
/* 0.5 */
size_t n = grafeo_node_count(db);
char* labels = grafeo_get_node_labels(db, id);
/* 0.6 */
size_t n;
if (grafeo_node_count(db, &n) != GRAFEO_OK) { /* the error */ }
char* labels;
GrafeoStatus status = grafeo_get_node_labels(db, id, &labels);
- Go:
NodeCount()andEdgeCount()return(int, error), an absent node or edge is the newErrNotFound(which still matchesErrDatabase), andDropVectorIndexreturns(bool, error), likeDropPropertyIndex. - C#:
NodeCountandEdgeCountthrow for a failed read,GrafeoGraph.GetNodeandGetEdgereturn null for an absent node or edge (they threw), andbyte,short,uint,ulong,Int128,UInt128andBigIntegerparameters arrive as integers (they were strings). - Dart:
getNode,getEdge,getNodeLabelsand the graph handle'sgetNodeandgetEdgereturn null for an absent node or edge (they threw), and aBigIntparameter arrives as an integer (it was a string). - Metrics: the Block-STM metrics are removed, also the
grafeo_block_stm_*counters on grafeo-server's/metrics; they always read 0. - Embeddings (
embed): an application that builds its own ONNX Runtime environment before Grafeo loads its first embedding model cannot load Grafeo models in the same process.
Command-Line Tool¶
- Standard output holds only results, and
--quiethides only status messages: progress and status lines (Creating full backup of ...,✓ Full backup created, the row count of a query) go to standard error, and-qleaves them out. Results (query rows,info,stats,schema,validate,index list,backup status) always print, so a script that counted on-qprinting nothing for a query now gets its rows. With--format jsonevery command prints JSON; a command that acts (backup,import,data dumpandload,wal checkpoint) prints a record of what it did. - Query results keep their values in JSON and CSV:
--format jsonwrites numbers, booleans, null, arrays and objects (it wrote"\"Alix\""and"30"), and CSV no longer doubles quotes and writes a date as its value. Fix parsers that worked around the old output. - The commands that only read open the database read-only, so
grafeo infoon a 0.5.x database no longer migrates it: use a command that writes, or open it read-write once. grafeo index listshows the name an index was created with (it made upidx_<label>_<property>), asGrafeoDB::list_indexes()does.- An error shows its whole chain of causes.
grafeo data loadruns in one transaction: an edge to a node not in the file, a duplicateidor a value it cannot read is an error naming the line, and nothing is loaded. A dump writes NaN and the infinities, and aDECIMALas{"Decimal": "1.50"}.grafeo backup restore --forcerefuses a target that is another directory or is open in another process.
Deprecated in 0.6, Removed in 0.7¶
Each of these still works in 0.6 and warns where your compiler or tooling can. Move to the replacement before you upgrade to 0.7.
| Deprecated | Use instead |
|---|---|
The public fields of Config | its getters, and its constructors and with_* methods |
Session::set_auto_commit, Session::auto_commit | begin_transaction to group writes; a write outside a transaction always commits on its own |
Config::adaptive, with_adaptive, without_adaptive, AdaptiveConfig | nothing: adaptive execution was never wired in |
StorageFormat::SingleFile, StorageFormat::WalDirectory | StorageFormat::Auto, the default: every database is a single file |
C grafeo_open_single_file, Go OpenSingleFile, Dart GrafeoDB.openSingleFile | grafeo_open, Open, GrafeoDB.open |
recompact() | compact() |
BackupCursor::log_sequence | wal_lsn |
InvertedIndex::with_tokenizer | InvertedIndex::with_options with a TokenizerKind |
RdfStore::insert_in_transaction, remove_in_transaction, commit_transaction, rollback_transaction, has_pending_ops, find_with_pending, contains_with_pending, find_in_graphs_with_pending | a session transaction |
The compact-store feature | nothing: every build that opens files reads a database compacted by 0.5.x |
The temporal feature | nothing: history is the per-database setting Config::with_history_retention, in every build |
The tiered-storage feature of grafeo-core and grafeo-common, and the lpg feature of grafeo-core, grafeo-adapters, grafeo-engine and grafeo-cli | nothing: every build has the labeled property graph store, and the store tiered-storage shaped is gone |