diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index e312807286e..3d3284d480d 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -53,6 +53,7 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima * Modified HTTP API to expect gremlin-lang strings for parameters and update all GLVs to send requests in new format. * Added string parameter parsing to `GremlinServer` to prevent traversal injection and excessive nesting depths. * Modified all GLVs to detect unsupported types in `GremlinLang` and throw consistent error for that case. +* Added GraphBinary 4.0 `Graph` (`0x10`) serializer/deserializer to `gremlin-javascript`, `gremlin-dotnet`, and `gremlin-go` so that `subgraph()` results are returned as a detached `Graph` data container. [[release-4-0-0-beta-2]] === TinkerPop 4.0.0-beta.2 (April 1, 2026) diff --git a/docs/src/reference/gremlin-variants.asciidoc b/docs/src/reference/gremlin-variants.asciidoc index 521a7017bf7..7b164f03c28 100644 --- a/docs/src/reference/gremlin-variants.asciidoc +++ b/docs/src/reference/gremlin-variants.asciidoc @@ -561,6 +561,12 @@ that can be used to fulfill the `gremlingo.Set` interface if desired. * Go does not support ordered maps natively as the built-in `map` type does not guarantee iteration order. Traversal results which contain maps may not preserve original ordering when deserialized into Go's native map types. +* The `subgraph()`-step returns a detached `*Graph` data container exposing +`Vertices map[interface{}]*Vertex` and `Edges map[interface{}]*Edge`. The result is not a live `Graph` instance: +mutating the maps has no effect on the source graph, and it cannot be passed to `traversal().with(...)`. To +re-query subgraph elements against the original graph, extract their `Id` and use `g.V(id)` / `g.E(id)` on the +original `GraphTraversalSource`. + [[gremlin-go-examples]] === Application Examples @@ -2035,9 +2041,11 @@ exact type sent to the server — see <>. signed range are unsuffixed (Int), integers beyond that up to `Number.MAX_SAFE_INTEGER` use the `L` suffix (Long), non-integer numbers and integers beyond the safe range use the `D` suffix (Double), and `BigInt` values use the `N` suffix (BigInteger). -* The `subgraph()`-step is not supported by any variant that is not running on the Java Virtual Machine as there is -no `Graph` instance to deserialize a result into on the client-side. A workaround is to replace the step with -`aggregate(local)` and then convert those results to something the client can use locally. +* The `subgraph()`-step returns a detached `Graph` data container exposing +`vertices: Map` and `edges: Map`. The result is not a live `Graph` instance: mutating the +collections has no effect on the source graph, and it cannot be passed to `traversal().with(...)`. To re-query +subgraph elements against the original graph, extract their `id` and use `g.V(id)` / `g.E(id)` on the original +`GraphTraversalSource`. [[gremlin-javascript-examples]] === Application Examples @@ -2451,9 +2459,11 @@ anchor:gremlin-net-limitations[] [[gremlin-dotnet-limitations]] === Limitations -* The `subgraph()`-step is not supported by any variant that is not running on the Java Virtual Machine as there is -no `Graph` instance to deserialize a result into on the client-side. A workaround is to replace the step with -`aggregate(local)` and then convert those results to something the client can use locally. +* The `subgraph()`-step returns a detached `Graph` data container exposing +`Vertices: IDictionary` and `Edges: IDictionary`. The result is not a live `Graph` +instance: mutating the collections has no effect on the source graph, and it cannot be passed to +`traversal().with(...)`. To re-query subgraph elements against the original graph, extract their `Id` and use +`g.V(id)` / `g.E(id)` on the original `GraphTraversalSource`. * `DateTimeOffset` cannot represent the extreme values of Gremlin's `OffsetDateTime` maximum and minimum, so offset date-time values at those boundaries will fail to deserialize. * Gremlin's `Duration` type has a much larger range than C#'s `TimeSpan`, so extreme duration values (such as @@ -3016,9 +3026,10 @@ and `timedelta`. * In Gremlin, 1 isn't equal to the boolean true value and 0 isn't equal to the boolean false value, but they are equal in Python. This means that in `gremlin-python` if these values are in a `Set`, you will get a different behavior than what is intended by Gremlin, since it follows Python's behavior. -* The `subgraph()`-step is not supported by any variant that is not running on the Java Virtual Machine as there is -no `Graph` instance to deserialize a result into on the client-side. A workaround is to replace the step with -`aggregate(local)` and then convert those results to something the client can use locally. +* The `subgraph()`-step returns a detached `Graph` data container exposing `vertices: dict` and `edges: dict` +keyed by element id. The result is not a live `Graph` instance: mutating the dicts has no effect on the source +graph, and it cannot be passed to `traversal().with(...)`. To re-query subgraph elements against the original +graph, extract their `id` and use `g.V(id)` / `g.E(id)` on the original `GraphTraversalSource`. * Use of the aiohttp library in the default transport requires the use of asyncio's event loop to run the async functions. This can be an issue in situations where the application calling Gremlin-Python is already using an event loop. Certain types of event loops can be patched using nest-asyncio which allows Gremlin-Python to proceed without an error like diff --git a/docs/src/upgrade/release-4.x.x.asciidoc b/docs/src/upgrade/release-4.x.x.asciidoc index aa14264ca0a..2b4606444cc 100644 --- a/docs/src/upgrade/release-4.x.x.asciidoc +++ b/docs/src/upgrade/release-4.x.x.asciidoc @@ -489,6 +489,18 @@ unwrap(toInt(29)); // 29 unwrap('hello'); // 'hello' ---- +==== Subgraph Support in GLVs + +All GLVs now support the `subgraph()` step. Previously, calling `subgraph()` from a GLV produced an unknown-type error +because the variant could not interpret the `Graph` payload that the server returned. Applications can now extract a +portion of a remote graph as part of a normal traversal and inspect its vertices and edges directly from the client, +without having to re-issue queries to reconstruct the result. See: <>. + +In the GLVs, the result is a detached snapshot of the captured vertices and edges, not a traversable `Graph` instance. +It cannot be passed to `traversal().with(...)`, and mutating its collections has no effect on the source graph. To +re-query elements against the original graph, extract their ids and call `g.V(id)` or `g.E(id)` against the original +`GraphTraversalSource`. + === Upgrading for Providers ==== Graph System Providers diff --git a/gremlin-dotnet/src/Gremlin.Net/Structure/Graph.cs b/gremlin-dotnet/src/Gremlin.Net/Structure/Graph.cs index 3d3416f64fd..e843f15ae40 100644 --- a/gremlin-dotnet/src/Gremlin.Net/Structure/Graph.cs +++ b/gremlin-dotnet/src/Gremlin.Net/Structure/Graph.cs @@ -22,6 +22,7 @@ #endregion using System; +using System.Collections.Generic; using Gremlin.Net.Process.Traversal; namespace Gremlin.Net.Structure @@ -32,6 +33,16 @@ namespace Gremlin.Net.Structure /// public class Graph { + /// + /// Gets the instances contained in this , keyed by their id. + /// + public IDictionary Vertices { get; } = new Dictionary(); + + /// + /// Gets the instances contained in this , keyed by their id. + /// + public IDictionary Edges { get; } = new Dictionary(); + /// /// Generates a reusable instance. /// @@ -41,5 +52,11 @@ public GraphTraversalSource Traversal() { return new GraphTraversalSource(); } + + /// + public override string ToString() + { + return $"graph[vertices:{Vertices.Count} edges:{Edges.Count}]"; + } } } \ No newline at end of file diff --git a/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/DataType.cs b/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/DataType.cs index 9490a34302e..23a47558957 100644 --- a/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/DataType.cs +++ b/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/DataType.cs @@ -44,8 +44,7 @@ public class DataType : IEquatable public static readonly DataType Edge = new DataType(0x0D); public static readonly DataType Path = new DataType(0x0E); public static readonly DataType Property = new DataType(0x0F); - // Not yet implemented - // public static readonly DataType Graph = new DataType(0x10); + public static readonly DataType Graph = new DataType(0x10); public static readonly DataType Vertex = new DataType(0x11); public static readonly DataType VertexProperty = new DataType(0x12); public static readonly DataType Direction = new DataType(0x18); diff --git a/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/TypeSerializerRegistry.cs b/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/TypeSerializerRegistry.cs index 82421f14a13..958e8d5c883 100644 --- a/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/TypeSerializerRegistry.cs +++ b/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/TypeSerializerRegistry.cs @@ -48,6 +48,7 @@ public class TypeSerializerRegistry {typeof(float), SingleTypeSerializers.FloatSerializer}, {typeof(Guid), new UuidSerializer()}, {typeof(Edge), new EdgeSerializer()}, + {typeof(Graph), new GraphSerializer()}, {typeof(Path), new PathSerializer()}, {typeof(Property), new PropertySerializer()}, {typeof(Vertex), new VertexSerializer()}, @@ -80,6 +81,7 @@ public class TypeSerializerRegistry {DataType.Set, new SetSerializer, object>()}, {DataType.Uuid, new UuidSerializer()}, {DataType.Edge, new EdgeSerializer()}, + {DataType.Graph, new GraphSerializer()}, {DataType.Path, new PathSerializer()}, {DataType.Property, new PropertySerializer()}, {DataType.Vertex, new VertexSerializer()}, diff --git a/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/Types/GraphSerializer.cs b/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/Types/GraphSerializer.cs new file mode 100644 index 00000000000..e40f7db8687 --- /dev/null +++ b/gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/Types/GraphSerializer.cs @@ -0,0 +1,225 @@ +#region License + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +#endregion + +using System; +using System.Collections; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Threading; +using System.Threading.Tasks; + +namespace Gremlin.Net.Structure.IO.GraphBinary4.Types +{ + /// + /// A serializer for GraphBinary. The wire format is a count-prefixed + /// list of vertices (each with their vertex properties and meta-properties), followed by a + /// count-prefixed list of edges (each with their properties). Vertex/edge labels are written + /// as a single-element list, parent placeholders are written as null. + /// + public class GraphSerializer : SimpleTypeSerializer + { + /// + /// Initializes a new instance of the class. + /// + public GraphSerializer() : base(DataType.Graph) + { + } + + /// + protected override async Task WriteValueAsync(Graph value, Stream stream, GraphBinaryWriter writer, + CancellationToken cancellationToken = default) + { + await writer.WriteNonNullableValueAsync(value.Vertices.Count, stream, cancellationToken) + .ConfigureAwait(false); + foreach (var vertex in value.Vertices.Values) + { + await WriteVertexAsync(vertex, stream, writer, cancellationToken).ConfigureAwait(false); + } + + await writer.WriteNonNullableValueAsync(value.Edges.Count, stream, cancellationToken) + .ConfigureAwait(false); + foreach (var edge in value.Edges.Values) + { + await WriteEdgeAsync(edge, stream, writer, cancellationToken).ConfigureAwait(false); + } + } + + private static async Task WriteVertexAsync(Vertex vertex, Stream stream, GraphBinaryWriter writer, + CancellationToken cancellationToken) + { + await writer.WriteAsync(vertex.Id, stream, cancellationToken).ConfigureAwait(false); + await writer.WriteNonNullableValueAsync(new List { vertex.Label }, stream, cancellationToken) + .ConfigureAwait(false); + + var vertexProperties = AsList(vertex.Properties); + await writer.WriteNonNullableValueAsync(vertexProperties.Count, stream, cancellationToken) + .ConfigureAwait(false); + foreach (var vp in vertexProperties) + { + await writer.WriteAsync(vp.Id, stream, cancellationToken).ConfigureAwait(false); + await writer.WriteNonNullableValueAsync(new List { vp.Label }, stream, cancellationToken) + .ConfigureAwait(false); + await writer.WriteAsync((object?)vp.Value, stream, cancellationToken).ConfigureAwait(false); + + // placeholder for the parent vertex + await writer.WriteAsync(null, stream, cancellationToken).ConfigureAwait(false); + + var metaProperties = AsList(vp.Properties); + await writer.WriteNonNullableValueAsync(metaProperties, stream, cancellationToken) + .ConfigureAwait(false); + } + } + + private static async Task WriteEdgeAsync(Edge edge, Stream stream, GraphBinaryWriter writer, + CancellationToken cancellationToken) + { + await writer.WriteAsync(edge.Id, stream, cancellationToken).ConfigureAwait(false); + await writer.WriteNonNullableValueAsync(new List { edge.Label }, stream, cancellationToken) + .ConfigureAwait(false); + + await writer.WriteAsync(edge.InV.Id, stream, cancellationToken).ConfigureAwait(false); + // placeholder for the in-vertex label (always null in this context) + await writer.WriteAsync(null, stream, cancellationToken).ConfigureAwait(false); + + await writer.WriteAsync(edge.OutV.Id, stream, cancellationToken).ConfigureAwait(false); + // placeholder for the out-vertex label (always null in this context) + await writer.WriteAsync(null, stream, cancellationToken).ConfigureAwait(false); + + // placeholder for the parent (never present) + await writer.WriteAsync(null, stream, cancellationToken).ConfigureAwait(false); + + var edgeProperties = AsList(edge.Properties); + await writer.WriteNonNullableValueAsync(edgeProperties, stream, cancellationToken) + .ConfigureAwait(false); + } + + /// + protected override async Task ReadValueAsync(Stream stream, GraphBinaryReader reader, + CancellationToken cancellationToken = default) + { + var graph = new Graph(); + + var vertexCount = + (int)await reader.ReadNonNullableValueAsync(stream, cancellationToken).ConfigureAwait(false); + for (var i = 0; i < vertexCount; i++) + { + var vertex = await ReadVertexAsync(stream, reader, cancellationToken).ConfigureAwait(false); + if (vertex.Id != null) + { + graph.Vertices[vertex.Id] = vertex; + } + } + + var edgeCount = + (int)await reader.ReadNonNullableValueAsync(stream, cancellationToken).ConfigureAwait(false); + for (var i = 0; i < edgeCount; i++) + { + var edge = await ReadEdgeAsync(graph, stream, reader, cancellationToken).ConfigureAwait(false); + if (edge.Id != null) + { + graph.Edges[edge.Id] = edge; + } + } + + return graph; + } + + private static async Task ReadVertexAsync(Stream stream, GraphBinaryReader reader, + CancellationToken cancellationToken) + { + var vId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + var vLabelList = (List)await reader + .ReadNonNullableValueAsync>(stream, cancellationToken).ConfigureAwait(false); + var vLabel = vLabelList.Count > 0 ? vLabelList[0] ?? "" : ""; + + var vpCount = (int)await reader.ReadNonNullableValueAsync(stream, cancellationToken) + .ConfigureAwait(false); + var vertexProperties = new List(vpCount); + for (var j = 0; j < vpCount; j++) + { + var vpId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + var vpLabelList = (List)await reader + .ReadNonNullableValueAsync>(stream, cancellationToken).ConfigureAwait(false); + var vpLabel = vpLabelList.Count > 0 ? vpLabelList[0] ?? "" : ""; + var vpValue = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + + // discard the parent vertex placeholder + await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + + var metaProps = await reader.ReadNonNullableValueAsync>(stream, cancellationToken) + .ConfigureAwait(false); + var metaPropsArray = (metaProps as List)?.ToArray() ?? Array.Empty(); + + vertexProperties.Add(new VertexProperty(vpId, vpLabel, vpValue, null, metaPropsArray)); + } + + return new Vertex(vId, vLabel, vertexProperties.Cast().ToArray()); + } + + private static async Task ReadEdgeAsync(Graph graph, Stream stream, GraphBinaryReader reader, + CancellationToken cancellationToken) + { + var eId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + var eLabelList = (List)await reader + .ReadNonNullableValueAsync>(stream, cancellationToken).ConfigureAwait(false); + var eLabel = eLabelList.Count > 0 ? eLabelList[0] ?? "" : ""; + + var inVId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + // discard the in-vertex label placeholder (always null on the wire) + await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + var outVId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + // discard the out-vertex label placeholder (always null on the wire) + await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + // discard the parent placeholder + await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); + + var edgeProps = await reader.ReadNonNullableValueAsync>(stream, cancellationToken) + .ConfigureAwait(false); + var edgePropsArray = (edgeProps as List)?.ToArray() ?? Array.Empty(); + + var inVertex = ResolveVertex(graph, inVId); + var outVertex = ResolveVertex(graph, outVId); + + return new Edge(eId, outVertex, eLabel, inVertex, edgePropsArray); + } + + private static Vertex ResolveVertex(Graph graph, object? vertexId) + { + if (vertexId != null && graph.Vertices.TryGetValue(vertexId, out var existing)) + { + return existing; + } + return new Vertex(vertexId, ""); + } + + private static List AsList(IEnumerable? source) + { + if (source == null) + { + return new List(); + } + return source.Cast().ToList(); + } + } +} diff --git a/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/CommonSteps.cs b/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/CommonSteps.cs index 790e412dcf3..2c42d1a8bed 100644 --- a/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/CommonSteps.cs +++ b/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/CommonSteps.cs @@ -216,6 +216,11 @@ public void IterateNext() case object[] arrayResult: _result = arrayResult; return; + case Graph graphResult: + // Graph is a container of vertices/edges but not iterable itself; wrap in a + // single-element array so assertions like AssertSubgraphStructure can find it. + _result = new object?[] { graphResult }; + return; case IEnumerable enumerableResult: _result = enumerableResult.Cast().ToArray(); return; @@ -297,7 +302,60 @@ public void AssertTreeStructure(string expectedTree) [Then("the result should be a subgraph with the following")] public void AssertSubgraphStructure(DataTable? table = null) { + AssertThatNoErrorWasThrown(); + + // The Cap step yields a single Graph instance as the only result. + Assert.NotNull(_result); + var sg = Assert.IsType(_result![0]); + + if (table == null) + { + return; + } + + var rows = table.Rows.ToArray(); + var columnName = rows[0].Cells.First().Value; + var assertingVertices = columnName == "vertices"; + + if (assertingVertices) + { + var expectedVertices = rows.Skip(1) + .Select(r => (Vertex)ParseValue(r.Cells.First().Value, _graphName!)!) + .ToList(); + Assert.Equal(expectedVertices.Count, sg.Vertices.Count); + foreach (var expected in expectedVertices) + { + Assert.NotNull(expected.Id); + Assert.True(sg.Vertices.ContainsKey(expected.Id!), + $"Expected subgraph to contain vertex with id {expected.Id}"); + var actual = sg.Vertices[expected.Id!]; + Assert.Equal(expected.Label, actual.Label); + + var variableKey = actual.Label == "person" ? "age" : "lang"; + Assert.Equal(expected.Property("name")?.Value, actual.Property("name")?.Value); + Assert.Equal(expected.Property(variableKey)?.Value, actual.Property(variableKey)?.Value); + } + } + else + { + var expectedEdges = rows.Skip(1) + .Select(r => (Edge)ParseValue(r.Cells.First().Value, _graphName!)!) + .ToList(); + Assert.Equal(expectedEdges.Count, sg.Edges.Count); + + foreach (var expected in expectedEdges) + { + Assert.NotNull(expected.Id); + Assert.True(sg.Edges.ContainsKey(expected.Id!), + $"Expected subgraph to contain edge with id {expected.Id}"); + var actual = sg.Edges[expected.Id!]; + Assert.Equal(expected.Label, actual.Label); + Assert.Equal(expected.Property("weight")?.Value, actual.Property("weight")?.Value); + Assert.Equal(expected.OutV.Id, actual.OutV.Id); + Assert.Equal(expected.InV.Id, actual.InV.Id); + } + } } [Then("the result should be (\\w+)\\s*")] diff --git a/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/GherkinTestRunner.cs b/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/GherkinTestRunner.cs index dd8f32fbca5..1754d7084ad 100644 --- a/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/GherkinTestRunner.cs +++ b/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/GherkinTestRunner.cs @@ -47,8 +47,6 @@ public class GherkinTestRunner new Dictionary { // Add here the name of scenarios to ignore and the reason, e.g.: - {"g_VX1X_outEXknowsX_subgraphXsgX_name_capXsgX", IgnoreReason.SubgraphStepNotSupported}, - {"g_V_repeatXbothEXcreatedX_subgraphXsgX_outVX_timesX5X_name_dedup_capXsgX", IgnoreReason.SubgraphStepNotSupported}, {"g_withStrategiesXProductiveByStrategyX_V_group_byXageX", IgnoreReason.NullKeysInMapNotSupported}, {"g_withStrategiesXProductiveByStrategyX_V_groupCount_byXageX", IgnoreReason.NullKeysInMapNotSupported}, {"g_withStrategiesXProductiveByStrategyX_V_group_byXageX_byXnameX", IgnoreReason.NullKeysInMapNotSupported}, @@ -127,12 +125,6 @@ public void RunGherkinBasedTests(IMessageSerializer messageSerializer) continue; } - if (scenario.Tags.Concat(feature.Tags).Any(t => t.Name == "@StepSubgraph")) - { - failedSteps.Add(scenario.Steps.First(), new IgnoreException(IgnoreReason.SubgraphStepNotSupported)); - continue; - } - StepBlock? currentStep = null; StepDefinition? stepDefinition = null; foreach (var step in scenario.Steps) diff --git a/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/Gremlin.cs b/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/Gremlin.cs index 448457ee970..7a0eae5b24b 100644 --- a/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/Gremlin.cs +++ b/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/Gremlin.cs @@ -1021,6 +1021,7 @@ private static IDictionary, ITraversal>> {(g,p) =>g.V().HasLabel("person").Values("age").AsString()}}, {"g_V_hasLabelXpersonX_valuesXageX_order_fold_asStringXlocalX", new List, ITraversal>> {(g,p) =>g.V().HasLabel("person").Values("age").Order().Fold().AsString(Scope.Local)}}, {"g_V_hasLabelXpersonX_valuesXageX_asString_concatX_years_oldX", new List, ITraversal>> {(g,p) =>g.V().HasLabel("person").Values("age").AsString().Concat(" years old")}}, + {"g_V_outEXknowsX_subgraphXsgX_capXsgX_asString", new List, ITraversal>> {(g,p) =>g.V().OutE("knows").Subgraph("sg").Cap("sg").AsString()}}, {"g_call", new List, ITraversal>> {(g,p) =>g.Call()}}, {"g_callXlistX", new List, ITraversal>> {(g,p) =>g.Call((string) "--list")}}, {"g_callXlistX_withXstring_stringX", new List, ITraversal>> {(g,p) =>g.Call((string) "--list").With("service", "tinker.search")}}, diff --git a/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/IgnoreException.cs b/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/IgnoreException.cs index 4647a52c694..70013910044 100644 --- a/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/IgnoreException.cs +++ b/gremlin-dotnet/test/Gremlin.Net.IntegrationTest/Gherkin/IgnoreException.cs @@ -62,11 +62,6 @@ public enum IgnoreReason /// NullPropertyValuesNotSupportedOnTestGraph, - /// - /// subgraph() is not supported yet - /// - SubgraphStepNotSupported, - /// /// tree() is not supported yet /// diff --git a/gremlin-dotnet/test/Gremlin.Net.UnitTest/Structure/IO/GraphBinary4/GraphSerializerTests.cs b/gremlin-dotnet/test/Gremlin.Net.UnitTest/Structure/IO/GraphBinary4/GraphSerializerTests.cs new file mode 100644 index 00000000000..9f3381e8482 --- /dev/null +++ b/gremlin-dotnet/test/Gremlin.Net.UnitTest/Structure/IO/GraphBinary4/GraphSerializerTests.cs @@ -0,0 +1,127 @@ +#region License + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +#endregion + +using System.IO; +using System.Linq; +using System.Threading.Tasks; +using Gremlin.Net.Structure; +using Gremlin.Net.Structure.IO.GraphBinary4; +using Xunit; + +namespace Gremlin.Net.UnitTest.Structure.IO.GraphBinary4 +{ + public class GraphSerializerTests + { + [Fact] + public async Task ShouldRoundTripGraphWithVerticesEdgesAndProperties() + { + // Round-trips a Graph with two vertices linked by a single edge; one vertex has a + // vertex property with a meta-property, and the edge carries a weight property. + var metaProperty = new Property("acl", "public"); + var nameProperty = new VertexProperty(4, "name", "marko", null, new object[] { metaProperty }); + var v1 = new Vertex(1, "person", new object[] { nameProperty }); + var v2 = new Vertex(2, "person"); + var weightProperty = new Property("weight", 0.5); + var e1 = new Edge(3, v1, "knows", v2, new object[] { weightProperty }); + + var graph = new Graph(); + graph.Vertices[v1.Id!] = v1; + graph.Vertices[v2.Id!] = v2; + graph.Edges[e1.Id!] = e1; + + var writer = new GraphBinaryWriter(); + var reader = new GraphBinaryReader(); + using var stream = new MemoryStream(); + await writer.WriteAsync(graph, stream); + stream.Position = 0; + var result = await reader.ReadAsync(stream); + + Assert.NotNull(result); + var deserialized = Assert.IsType(result); + Assert.Equal(2, deserialized.Vertices.Count); + Assert.Single(deserialized.Edges); + + var rv1 = deserialized.Vertices[1]; + Assert.Equal("person", rv1.Label); + var rvProperties = rv1.Properties.Cast().ToList(); + Assert.Single(rvProperties); + var rvp1 = rvProperties[0]; + Assert.Equal(4, rvp1.Id); + Assert.Equal("name", rvp1.Label); + Assert.Equal("marko", (string?)rvp1.Value); + var metaProps = rvp1.Properties.Cast().ToList(); + Assert.Single(metaProps); + Assert.Equal("acl", metaProps[0].Key); + Assert.Equal("public", (string?)metaProps[0].Value); + + Assert.True(deserialized.Vertices.ContainsKey(2)); + Assert.Equal("person", deserialized.Vertices[2].Label); + + var re1 = deserialized.Edges[3]; + Assert.Equal("knows", re1.Label); + Assert.Equal(1, re1.OutV.Id); + Assert.Equal(2, re1.InV.Id); + var edgeProps = re1.Properties.Cast().ToList(); + Assert.Single(edgeProps); + Assert.Equal("weight", edgeProps[0].Key); + Assert.Equal(0.5, (double?)edgeProps[0].Value); + } + + [Fact] + public async Task ShouldRoundTripEmptyGraph() + { + var graph = new Graph(); + + var writer = new GraphBinaryWriter(); + var reader = new GraphBinaryReader(); + using var stream = new MemoryStream(); + await writer.WriteAsync(graph, stream); + stream.Position = 0; + var result = await reader.ReadAsync(stream); + + Assert.NotNull(result); + var deserialized = Assert.IsType(result); + Assert.Empty(deserialized.Vertices); + Assert.Empty(deserialized.Edges); + } + + [Fact] + public void ToStringShouldRenderEmptyGraphCounts() + { + var graph = new Graph(); + Assert.Equal("graph[vertices:0 edges:0]", graph.ToString()); + } + + [Fact] + public void ToStringShouldRenderVerticesAndEdgesCounts() + { + var v1 = new Vertex(1, "person"); + var v2 = new Vertex(2, "person"); + var graph = new Graph(); + graph.Vertices[v1.Id!] = v1; + graph.Vertices[v2.Id!] = v2; + graph.Edges[3] = new Edge(3, v1, "knows", v2); + Assert.Equal("graph[vertices:2 edges:1]", graph.ToString()); + } + } +} diff --git a/gremlin-go/driver/cucumber/cucumberSteps_test.go b/gremlin-go/driver/cucumber/cucumberSteps_test.go index 2d72befcb76..495d56fd408 100644 --- a/gremlin-go/driver/cucumber/cucumberSteps_test.go +++ b/gremlin-go/driver/cucumber/cucumberSteps_test.go @@ -578,6 +578,86 @@ func (tg *tinkerPopGraph) theResultShouldBeEmpty() error { return nil } +// theResultShouldBeASubgraphWithTheFollowing asserts that the most recent +// traversal result is a *gremlingo.Graph data container whose Vertices or +// Edges map (selected by the data-table header) matches the expected rows. +func (tg *tinkerPopGraph) theResultShouldBeASubgraphWithTheFollowing(table *godog.Table) error { + if len(tg.result) == 0 { + return errors.New("no result to assert against") + } + + res, ok := tg.result[0].(*gremlingo.Result) + if !ok { + return fmt.Errorf("expected first result to be *gremlingo.Result, got %T", tg.result[0]) + } + sg, ok := res.GetInterface().(*gremlingo.Graph) + if !ok { + return fmt.Errorf("expected result to be *gremlingo.Graph, got %T", res.GetInterface()) + } + + if table == nil || len(table.Rows) == 0 { + return nil + } + + header := table.Rows[0].Cells[0].Value + expectedRows := table.Rows[1:] + + switch header { + case "vertices": + if len(sg.Vertices) != len(expectedRows) { + return fmt.Errorf("subgraph vertex count mismatch: expected %d, got %d", + len(expectedRows), len(sg.Vertices)) + } + for _, row := range expectedRows { + parsed := parseValue(row.Cells[0].Value, tg.graphName) + expected, ok := parsed.(*gremlingo.Vertex) + if !ok { + return fmt.Errorf("could not parse expected vertex %q (got %T)", row.Cells[0].Value, parsed) + } + actual, ok := sg.Vertices[expected.Id] + if !ok { + return fmt.Errorf("subgraph is missing vertex with id %v", expected.Id) + } + if actual.Label != expected.Label { + return fmt.Errorf("vertex %v: expected label %q, got %q", + expected.Id, expected.Label, actual.Label) + } + } + case "edges": + if len(sg.Edges) != len(expectedRows) { + return fmt.Errorf("subgraph edge count mismatch: expected %d, got %d", + len(expectedRows), len(sg.Edges)) + } + for _, row := range expectedRows { + parsed := parseValue(row.Cells[0].Value, tg.graphName) + expected, ok := parsed.(*gremlingo.Edge) + if !ok { + return fmt.Errorf("could not parse expected edge %q (got %T)", row.Cells[0].Value, parsed) + } + actual, ok := sg.Edges[expected.Id] + if !ok { + return fmt.Errorf("subgraph is missing edge with id %v", expected.Id) + } + if actual.Label != expected.Label { + return fmt.Errorf("edge %v: expected label %q, got %q", + expected.Id, expected.Label, actual.Label) + } + if actual.OutV.Id != expected.OutV.Id { + return fmt.Errorf("edge %v: expected outV.Id %v, got %v", + expected.Id, expected.OutV.Id, actual.OutV.Id) + } + if actual.InV.Id != expected.InV.Id { + return fmt.Errorf("edge %v: expected inV.Id %v, got %v", + expected.Id, expected.InV.Id, actual.InV.Id) + } + } + default: + return fmt.Errorf("unknown subgraph assertion header: %q", header) + } + + return nil +} + func (tg *tinkerPopGraph) theResultShouldBe(characterizedAs string, table *godog.Table) error { ordered := characterizedAs == "ordered" // For comparing ordered gremlingo.SimpleSet case. @@ -1016,6 +1096,7 @@ func InitializeScenario(ctx *godog.ScenarioContext) { ctx.Step(`^the graph initializer of$`, tg.theGraphInitializerOf) ctx.Step(`^the graph should return (\d+) for count of "(.+)"$`, tg.theGraphShouldReturnForCountOf) ctx.Step(`^the result should be empty$`, tg.theResultShouldBeEmpty) + ctx.Step(`^the result should be a subgraph with the following$`, tg.theResultShouldBeASubgraphWithTheFollowing) ctx.Step(`^the result should be (o\w+)$`, tg.theResultShouldBe) ctx.Step(`^the result should be (u\w+)$`, tg.theResultShouldBe) ctx.Step(`^the result should have a count of (\d+)$`, tg.theResultShouldHaveACountOf) @@ -1051,7 +1132,7 @@ func TestCucumberFeatures(t *testing.T) { TestSuiteInitializer: InitializeTestSuite, ScenarioInitializer: InitializeScenario, Options: &godog.Options{ - Tags: "~@GraphComputerOnly && ~@AllowNullPropertyValues && ~@StepSubgraph && ~@StepTree && ~@StepWrite && ~@DataChar", + Tags: "~@GraphComputerOnly && ~@AllowNullPropertyValues && ~@StepTree && ~@StepWrite && ~@DataChar", Format: "pretty", Paths: []string{getEnvOrDefaultString("CUCUMBER_FEATURE_FOLDER", "../../../gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features")}, TestingT: t, // Testing instance that will run subtests. diff --git a/gremlin-go/driver/cucumber/gremlin.go b/gremlin-go/driver/cucumber/gremlin.go index 2c821d785d2..574fdb45fc3 100644 --- a/gremlin-go/driver/cucumber/gremlin.go +++ b/gremlin-go/driver/cucumber/gremlin.go @@ -991,6 +991,7 @@ var translationMap = map[string][]func(g *gremlingo.GraphTraversalSource, p map[ "g_V_hasLabelXpersonX_valuesXageX_asString": {func(g *gremlingo.GraphTraversalSource, p map[string]interface{}) *gremlingo.GraphTraversal {return g.V().HasLabel("person").Values("age").AsString()}}, "g_V_hasLabelXpersonX_valuesXageX_order_fold_asStringXlocalX": {func(g *gremlingo.GraphTraversalSource, p map[string]interface{}) *gremlingo.GraphTraversal {return g.V().HasLabel("person").Values("age").Order().Fold().AsString(gremlingo.Scope.Local)}}, "g_V_hasLabelXpersonX_valuesXageX_asString_concatX_years_oldX": {func(g *gremlingo.GraphTraversalSource, p map[string]interface{}) *gremlingo.GraphTraversal {return g.V().HasLabel("person").Values("age").AsString().Concat(" years old")}}, + "g_V_outEXknowsX_subgraphXsgX_capXsgX_asString": {func(g *gremlingo.GraphTraversalSource, p map[string]interface{}) *gremlingo.GraphTraversal {return g.V().OutE("knows").Subgraph("sg").Cap("sg").AsString()}}, "g_call": {func(g *gremlingo.GraphTraversalSource, p map[string]interface{}) *gremlingo.GraphTraversal {return g.Call()}}, "g_callXlistX": {func(g *gremlingo.GraphTraversalSource, p map[string]interface{}) *gremlingo.GraphTraversal {return g.Call("--list")}}, "g_callXlistX_withXstring_stringX": {func(g *gremlingo.GraphTraversalSource, p map[string]interface{}) *gremlingo.GraphTraversal {return g.Call("--list").With("service", "tinker.search")}}, diff --git a/gremlin-go/driver/graph.go b/gremlin-go/driver/graph.go index a59b41e9bcc..981a7fc2d30 100644 --- a/gremlin-go/driver/graph.go +++ b/gremlin-go/driver/graph.go @@ -26,7 +26,24 @@ import ( ) // Graph is used to store the graph. +// In-memory collections of vertices and edges, typically produced by a subgraph() +// traversal. Maps are keyed by element id. type Graph struct { + Vertices map[interface{}]*Vertex + Edges map[interface{}]*Edge +} + +// NewGraph creates a new empty Graph with initialized Vertices and Edges maps. +func NewGraph() *Graph { + return &Graph{ + Vertices: make(map[interface{}]*Vertex), + Edges: make(map[interface{}]*Edge), + } +} + +// String returns the string representation of the graph. +func (g *Graph) String() string { + return fmt.Sprintf("graph[vertices:%d edges:%d]", len(g.Vertices), len(g.Edges)) } // Element is the base structure for both Vertex and Edge. diff --git a/gremlin-go/driver/graphBinaryDeserializer.go b/gremlin-go/driver/graphBinaryDeserializer.go index 5939dbbda07..f13acf9eac7 100644 --- a/gremlin-go/driver/graphBinaryDeserializer.go +++ b/gremlin-go/driver/graphBinaryDeserializer.go @@ -243,6 +243,8 @@ func (d *GraphBinaryDeserializer) readValue(dt dataType, flag byte) (interface{} return d.readVertex(true) case edgeType: return d.readEdge() + case graphType: + return d.readGraph() case pathType: return d.readPath() case propertyType: @@ -416,6 +418,191 @@ func (d *GraphBinaryDeserializer) readEdge() (*Edge, error) { return e, nil } +func (d *GraphBinaryDeserializer) readGraph() (*Graph, error) { + graph := NewGraph() + + // {vertex_count} value-only int32 + vertexCount, err := d.readInt32() + if err != nil { + return nil, err + } + + for i := int32(0); i < vertexCount; i++ { + // {id} fully-qualified + vId, err := d.ReadFullyQualified() + if err != nil { + return nil, err + } + + // {labels} list value-only, take first element + vLabels, err := d.readList(false) + if err != nil { + return nil, err + } + labelSlice, ok := vLabels.([]interface{}) + if !ok || len(labelSlice) == 0 { + return nil, newError(err0404ReadNullTypeError) + } + vLabel, ok := labelSlice[0].(string) + if !ok { + return nil, newError(err0404ReadNullTypeError) + } + + v := &Vertex{Element: Element{Id: vId, Label: vLabel}} + + // {vp_count} value-only int32 + vpCount, err := d.readInt32() + if err != nil { + return nil, err + } + + vps := make([]*VertexProperty, 0, vpCount) + for j := int32(0); j < vpCount; j++ { + // {vp_id} fully-qualified + vpId, err := d.ReadFullyQualified() + if err != nil { + return nil, err + } + + // {vp_label} list value-only, take first element + vpLabels, err := d.readList(false) + if err != nil { + return nil, err + } + vpLabelSlice, ok := vpLabels.([]interface{}) + if !ok || len(vpLabelSlice) == 0 { + return nil, newError(err0404ReadNullTypeError) + } + vpLabel, ok := vpLabelSlice[0].(string) + if !ok { + return nil, newError(err0404ReadNullTypeError) + } + + // {vp_value} fully-qualified + vpValue, err := d.ReadFullyQualified() + if err != nil { + return nil, err + } + + // {parent} fully-qualified null placeholder — discard + if _, err := d.ReadFullyQualified(); err != nil { + return nil, err + } + + // {meta_props} value-only list + metaProps, err := d.readList(false) + if err != nil { + return nil, err + } + + vp := &VertexProperty{ + Element: Element{Id: vpId, Label: vpLabel}, + Key: vpLabel, + Value: vpValue, + Vertex: *v, + } + vp.Properties = make([]interface{}, 0) + if metaProps != nil { + vp.Properties = metaProps + } + vps = append(vps, vp) + } + // Store vertex properties on the Vertex; match the existing convention + // of storing as []interface{} (see readVertex above). + if vpCount > 0 { + vpList := make([]interface{}, len(vps)) + for k, vp := range vps { + vpList[k] = vp + } + v.Properties = vpList + } else { + v.Properties = make([]interface{}, 0) + } + + graph.Vertices[vId] = v + } + + // {edge_count} value-only int32 + edgeCount, err := d.readInt32() + if err != nil { + return nil, err + } + + for i := int32(0); i < edgeCount; i++ { + // {id} fully-qualified + eId, err := d.ReadFullyQualified() + if err != nil { + return nil, err + } + + // {labels} list value-only, take first element + eLabels, err := d.readList(false) + if err != nil { + return nil, err + } + labelSlice, ok := eLabels.([]interface{}) + if !ok || len(labelSlice) == 0 { + return nil, newError(err0404ReadNullTypeError) + } + eLabel, ok := labelSlice[0].(string) + if !ok { + return nil, newError(err0404ReadNullTypeError) + } + + // {inV_id} fully-qualified + inVId, err := d.ReadFullyQualified() + if err != nil { + return nil, err + } + // {inV_label} fully-qualified null placeholder — discard + if _, err := d.ReadFullyQualified(); err != nil { + return nil, err + } + // {outV_id} fully-qualified + outVId, err := d.ReadFullyQualified() + if err != nil { + return nil, err + } + // {outV_label} fully-qualified null placeholder — discard + if _, err := d.ReadFullyQualified(); err != nil { + return nil, err + } + // {parent} fully-qualified null placeholder — discard + if _, err := d.ReadFullyQualified(); err != nil { + return nil, err + } + + // {props} value-only list + props, err := d.readList(false) + if err != nil { + return nil, err + } + + // Reuse previously-read vertex instances if present, else build stand-ins. + inV, ok := graph.Vertices[inVId] + if !ok { + inV = &Vertex{Element: Element{Id: inVId}} + } + outV, ok := graph.Vertices[outVId] + if !ok { + outV = &Vertex{Element: Element{Id: outVId}} + } + + e := &Edge{ + Element: Element{Id: eId, Label: eLabel}, + InV: *inV, + OutV: *outV, + } + e.Properties = make([]interface{}, 0) + if props != nil { + e.Properties = props + } + graph.Edges[eId] = e + } + + return graph, nil +} + func (d *GraphBinaryDeserializer) readPath() (*Path, error) { labels, err := d.ReadFullyQualified() if err != nil { diff --git a/gremlin-go/driver/graphBinarySerializer.go b/gremlin-go/driver/graphBinarySerializer.go index c473bb880a4..f169b7fde68 100644 --- a/gremlin-go/driver/graphBinarySerializer.go +++ b/gremlin-go/driver/graphBinarySerializer.go @@ -51,6 +51,7 @@ const ( edgeType dataType = 0x0d pathType dataType = 0x0e propertyType dataType = 0x0f + graphType dataType = 0x10 vertexType dataType = 0x11 vertexPropertyType dataType = 0x12 directionType dataType = 0x18 @@ -363,6 +364,146 @@ func pathWriter(value interface{}, w io.Writer, typeSerializer *graphBinaryTypeS return typeSerializer.write(p.Objects, w) } +// Format: {vertex_count}{vertices...}{edge_count}{edges...} +// Per vertex: {id}{labels:list}{vp_count}{vps...} +// Per vp: {id}{labels:list}{value}{parent=null}{meta_props:list} +// Per edge: {id}{labels:list}{inVId}{inVLabel=null}{outVId}{outVLabel=null}{parent=null}{props:list} +func graphWriter(value interface{}, w io.Writer, typeSerializer *graphBinaryTypeSerializer) error { + g := value.(*Graph) + + // vertex_count (value-only int32) + if err := typeSerializer.writeValue(int32(len(g.Vertices)), w, false); err != nil { + return err + } + + for _, v := range g.Vertices { + // {id} fully-qualified + if err := typeSerializer.write(v.Id, w); err != nil { + return err + } + // {labels} list value-only, 1 element + if err := typeSerializer.writeValue([1]string{v.Label}, w, false); err != nil { + return err + } + + // Collect vertex properties as []*VertexProperty regardless of how they are stored. + vps := asVertexProperties(v.Properties) + + // {vp_count} value-only int32 + if err := typeSerializer.writeValue(int32(len(vps)), w, false); err != nil { + return err + } + + for _, vp := range vps { + // {vp_id} fully-qualified + if err := typeSerializer.write(vp.Id, w); err != nil { + return err + } + // {vp_label} list value-only, 1 element + if err := typeSerializer.writeValue([1]string{vp.Label}, w, false); err != nil { + return err + } + // {vp_value} fully-qualified + if err := typeSerializer.write(vp.Value, w); err != nil { + return err + } + // {parent} fully-qualified null placeholder + if _, err := w.Write(nullBytes); err != nil { + return err + } + // {meta_props} value-only list + if err := typeSerializer.writeValue(asProperties(vp.Properties), w, false); err != nil { + return err + } + } + } + + // edge_count (value-only int32) + if err := typeSerializer.writeValue(int32(len(g.Edges)), w, false); err != nil { + return err + } + + for _, e := range g.Edges { + // {id} fully-qualified + if err := typeSerializer.write(e.Id, w); err != nil { + return err + } + // {labels} list value-only, 1 element + if err := typeSerializer.writeValue([1]string{e.Label}, w, false); err != nil { + return err + } + // {inV_id} fully-qualified + if err := typeSerializer.write(e.InV.Id, w); err != nil { + return err + } + // {inV_label} fully-qualified null placeholder + if _, err := w.Write(nullBytes); err != nil { + return err + } + // {outV_id} fully-qualified + if err := typeSerializer.write(e.OutV.Id, w); err != nil { + return err + } + // {outV_label} fully-qualified null placeholder + if _, err := w.Write(nullBytes); err != nil { + return err + } + // {parent} fully-qualified null placeholder + if _, err := w.Write(nullBytes); err != nil { + return err + } + // {props} value-only list + if err := typeSerializer.writeValue(asProperties(e.Properties), w, false); err != nil { + return err + } + } + + return nil +} + +// asVertexProperties coerces the interface{}-typed Properties field on a Vertex +// to a slice of *VertexProperty. Returns an empty slice if nothing matches. +func asVertexProperties(props interface{}) []*VertexProperty { + if props == nil { + return nil + } + if vps, ok := props.([]*VertexProperty); ok { + return vps + } + if list, ok := props.([]interface{}); ok { + out := make([]*VertexProperty, 0, len(list)) + for _, p := range list { + if vp, ok := p.(*VertexProperty); ok { + out = append(out, vp) + } + } + return out + } + return nil +} + +// asProperties coerces the interface{}-typed Properties field on a Vertex, +// VertexProperty, or Edge to a slice of *Property. Returns an empty slice if +// nothing matches. +func asProperties(props interface{}) []*Property { + if props == nil { + return nil + } + if ps, ok := props.([]*Property); ok { + return ps + } + if list, ok := props.([]interface{}); ok { + out := make([]*Property, 0, len(list)) + for _, p := range list { + if pp, ok := p.(*Property); ok { + out = append(out, pp) + } + } + return out + } + return nil +} + // Format: Same as List. // Mostly similar to listWriter with small changes func setWriter(value interface{}, w io.Writer, typeSerializer *graphBinaryTypeSerializer) error { @@ -454,6 +595,8 @@ func (serializer *graphBinaryTypeSerializer) getType(val interface{}) (dataType, return vertexType, nil case *Edge: return edgeType, nil + case *Graph: + return graphType, nil case *Property: return propertyType, nil case *VertexProperty: diff --git a/gremlin-go/driver/graphBinarySerializer_test.go b/gremlin-go/driver/graphBinarySerializer_test.go index 4403ea70c4c..fcf237147db 100644 --- a/gremlin-go/driver/graphBinarySerializer_test.go +++ b/gremlin-go/driver/graphBinarySerializer_test.go @@ -95,6 +95,17 @@ func TestGraphBinaryV4(t *testing.T) { _, _, err := serializer.getSerializerToWrite(nullType) assert.NotNil(t, err) }) + + t.Run("getType returns graphType for *Graph", func(t *testing.T) { + res, err := serializer.getType(NewGraph()) + assert.Nil(t, err) + assert.Equal(t, graphType, res) + }) + + t.Run("getWriter returns graphWriter for graphType", func(t *testing.T) { + _, err := serializer.getWriter(graphType) + assert.Nil(t, err) + }) }) t.Run("read-write tests", func(t *testing.T) { @@ -383,6 +394,179 @@ func TestGraphBinaryV4(t *testing.T) { } +// TestGraphSerializerRoundTrip verifies that GraphBinary serialization and +// deserialization round-trips a *Graph while preserving its vertices, edges, +// vertex-properties and properties. +func TestGraphSerializerRoundTrip(t *testing.T) { + t.Run("preserves vertices, edges, vertex-properties and properties", func(t *testing.T) { + graph := NewGraph() + + v1 := &Vertex{Element: Element{Id: int32(1), Label: "person"}} + v2 := &Vertex{Element: Element{Id: int32(2), Label: "person"}} + + // VertexProperty on v1 with a meta-property + vp1 := &VertexProperty{ + Element: Element{Id: int32(4), Label: "name"}, + Key: "name", + Value: "marko", + Vertex: *v1, + } + vp1.Properties = []interface{}{&Property{Key: "acl", Value: "public"}} + v1.Properties = []interface{}{vp1} + v2.Properties = []interface{}{} + + graph.Vertices[v1.Id] = v1 + graph.Vertices[v2.Id] = v2 + + // Edge v1 -knows-> v2 with weight property + e1 := &Edge{ + Element: Element{Id: int32(3), Label: "knows"}, + InV: *v2, + OutV: *v1, + } + e1.Properties = []interface{}{&Property{Key: "weight", Value: 0.5}} + graph.Edges[e1.Id] = e1 + + var buffer bytes.Buffer + serializer := graphBinaryTypeSerializer{newLogHandler(&defaultLogger{}, Error, language.English)} + + // Round-trip via fully-qualified write/read. + assert.Nil(t, serializer.write(graph, &buffer)) + + d := NewGraphBinaryDeserializer(bytes.NewReader(buffer.Bytes())) + out, err := d.ReadFullyQualified() + assert.Nil(t, err) + + rg, ok := out.(*Graph) + assert.True(t, ok, "expected *Graph result, got %T", out) + assert.Equal(t, 2, len(rg.Vertices)) + assert.Equal(t, 1, len(rg.Edges)) + + rv1 := rg.Vertices[int32(1)] + assert.NotNil(t, rv1) + assert.Equal(t, "person", rv1.Label) + rv1Props, _ := rv1.Properties.([]interface{}) + assert.Equal(t, 1, len(rv1Props)) + rvp1, _ := rv1Props[0].(*VertexProperty) + assert.NotNil(t, rvp1) + assert.Equal(t, "name", rvp1.Label) + assert.Equal(t, "marko", rvp1.Value) + + rvp1Meta, _ := rvp1.Properties.([]interface{}) + assert.Equal(t, 1, len(rvp1Meta)) + meta, _ := rvp1Meta[0].(*Property) + assert.NotNil(t, meta) + assert.Equal(t, "acl", meta.Key) + assert.Equal(t, "public", meta.Value) + + rv2 := rg.Vertices[int32(2)] + assert.NotNil(t, rv2) + assert.Equal(t, "person", rv2.Label) + + re1 := rg.Edges[int32(3)] + assert.NotNil(t, re1) + assert.Equal(t, "knows", re1.Label) + assert.Equal(t, int32(1), re1.OutV.Id) + assert.Equal(t, int32(2), re1.InV.Id) + + re1Props, _ := re1.Properties.([]interface{}) + assert.Equal(t, 1, len(re1Props)) + w, _ := re1Props[0].(*Property) + assert.NotNil(t, w) + assert.Equal(t, "weight", w.Key) + assert.Equal(t, 0.5, w.Value) + }) + + t.Run("handles empty graph", func(t *testing.T) { + graph := NewGraph() + + var buffer bytes.Buffer + serializer := graphBinaryTypeSerializer{newLogHandler(&defaultLogger{}, Error, language.English)} + assert.Nil(t, serializer.write(graph, &buffer)) + + d := NewGraphBinaryDeserializer(bytes.NewReader(buffer.Bytes())) + out, err := d.ReadFullyQualified() + assert.Nil(t, err) + + rg, ok := out.(*Graph) + assert.True(t, ok, "expected *Graph result, got %T", out) + assert.Equal(t, 0, len(rg.Vertices)) + assert.Equal(t, 0, len(rg.Edges)) + }) + + t.Run("handles vertices without properties and edges without properties", func(t *testing.T) { + graph := NewGraph() + + v1 := &Vertex{Element: Element{Id: int32(10), Label: "person"}} + v2 := &Vertex{Element: Element{Id: int32(20), Label: "software"}} + v1.Properties = []interface{}{} + v2.Properties = []interface{}{} + graph.Vertices[v1.Id] = v1 + graph.Vertices[v2.Id] = v2 + + e := &Edge{ + Element: Element{Id: int32(30), Label: "created"}, + InV: *v2, + OutV: *v1, + } + e.Properties = []interface{}{} + graph.Edges[e.Id] = e + + var buffer bytes.Buffer + serializer := graphBinaryTypeSerializer{newLogHandler(&defaultLogger{}, Error, language.English)} + assert.Nil(t, serializer.write(graph, &buffer)) + + d := NewGraphBinaryDeserializer(bytes.NewReader(buffer.Bytes())) + out, err := d.ReadFullyQualified() + assert.Nil(t, err) + + rg, _ := out.(*Graph) + assert.Equal(t, 2, len(rg.Vertices)) + assert.Equal(t, 1, len(rg.Edges)) + + re := rg.Edges[int32(30)] + assert.Equal(t, "created", re.Label) + assert.Equal(t, int32(10), re.OutV.Id) + assert.Equal(t, int32(20), re.InV.Id) + }) + + t.Run("handles string ids", func(t *testing.T) { + graph := NewGraph() + + v1 := &Vertex{Element: Element{Id: "a", Label: "person"}} + v2 := &Vertex{Element: Element{Id: "b", Label: "person"}} + v1.Properties = []interface{}{} + v2.Properties = []interface{}{} + graph.Vertices[v1.Id] = v1 + graph.Vertices[v2.Id] = v2 + + e := &Edge{ + Element: Element{Id: "e1", Label: "knows"}, + InV: *v2, + OutV: *v1, + } + e.Properties = []interface{}{} + graph.Edges[e.Id] = e + + var buffer bytes.Buffer + serializer := graphBinaryTypeSerializer{newLogHandler(&defaultLogger{}, Error, language.English)} + assert.Nil(t, serializer.write(graph, &buffer)) + + d := NewGraphBinaryDeserializer(bytes.NewReader(buffer.Bytes())) + out, err := d.ReadFullyQualified() + assert.Nil(t, err) + + rg, _ := out.(*Graph) + assert.Equal(t, 2, len(rg.Vertices)) + assert.Equal(t, 1, len(rg.Edges)) + + re := rg.Edges["e1"] + assert.NotNil(t, re) + assert.Equal(t, "a", re.OutV.Id) + assert.Equal(t, "b", re.InV.Id) + }) +} + // TestWriterErrorPropagation tests that errors from io.Writer are properly propagated // through the serialization chain. // Feature: serializer-writer-refactor, Property 4: Writer Error Propagation diff --git a/gremlin-go/driver/graph_test.go b/gremlin-go/driver/graph_test.go index 0890e01aaca..0de4a1bac97 100644 --- a/gremlin-go/driver/graph_test.go +++ b/gremlin-go/driver/graph_test.go @@ -56,6 +56,24 @@ func TestGraphStructureFunctions(t *testing.T) { assert.Equal(t, "p[property-Key->[0 1]]", p.String()) }) + t.Run("Test Graph.String() empty graph", func(t *testing.T) { + g := NewGraph() + assert.Equal(t, "graph[vertices:0 edges:0]", g.String()) + }) + + t.Run("Test Graph.String() with vertices and edges", func(t *testing.T) { + g := NewGraph() + v1 := &Vertex{Element: Element{Id: int32(1), Label: "person"}} + v2 := &Vertex{Element: Element{Id: int32(2), Label: "person"}} + g.Vertices[v1.Id] = v1 + g.Vertices[v2.Id] = v2 + g.Edges[int32(3)] = &Edge{ + Element: Element{Id: int32(3), Label: "knows"}, + InV: *v2, OutV: *v1, + } + assert.Equal(t, "graph[vertices:2 edges:1]", g.String()) + }) + s1 := NewSimpleSet("foo") s2 := NewSimpleSet("bar") s3 := NewSimpleSet("baz") diff --git a/gremlin-go/driver/serializer.go b/gremlin-go/driver/serializer.go index bd4a4262080..7b2266db2c7 100644 --- a/gremlin-go/driver/serializer.go +++ b/gremlin-go/driver/serializer.go @@ -220,6 +220,7 @@ func initSerializers() { propertyType: propertyWriter, vertexPropertyType: vertexPropertyWriter, pathType: pathWriter, + graphType: graphWriter, datetimeType: dateTimeWriter, durationType: durationWriter, directionType: enumWriter, diff --git a/gremlin-js/gremlin-javascript/lib/structure/graph.ts b/gremlin-js/gremlin-javascript/lib/structure/graph.ts index c8b86443601..8b918285121 100644 --- a/gremlin-js/gremlin-javascript/lib/structure/graph.ts +++ b/gremlin-js/gremlin-javascript/lib/structure/graph.ts @@ -23,10 +23,21 @@ /** * An "empty" graph object to server only as a reference. + * + * Holds in-memory collections of vertices and edges so that GraphBinary + * Graph (0x10) deserialization can return a usable data container. */ export class Graph { + readonly vertices: Map; + readonly edges: Map; + + constructor() { + this.vertices = new Map(); + this.edges = new Map(); + } + toString() { - return 'graph[]'; + return `graph[vertices:${this.vertices.size} edges:${this.edges.size}]`; } } diff --git a/gremlin-js/gremlin-javascript/lib/structure/io/binary/GraphBinary.js b/gremlin-js/gremlin-javascript/lib/structure/io/binary/GraphBinary.js index 4bd402b5960..7f917b9b198 100644 --- a/gremlin-js/gremlin-javascript/lib/structure/io/binary/GraphBinary.js +++ b/gremlin-js/gremlin-javascript/lib/structure/io/binary/GraphBinary.js @@ -56,6 +56,7 @@ import PathSerializer from './internals/PathSerializer.js'; import PropertySerializer from './internals/PropertySerializer.js'; import VertexSerializer from './internals/VertexSerializer.js'; import VertexPropertySerializer from './internals/VertexPropertySerializer.js'; +import GraphSerializer from './internals/GraphSerializer.js'; import BigIntegerSerializer from './internals/BigIntegerSerializer.js'; import ByteSerializer from './internals/ByteSerializer.js'; import BinarySerializer from './internals/BinarySerializer.js'; @@ -80,35 +81,35 @@ function createIoc(anySerializerOptions) { ioc.serializers = {}; - ioc.intSerializer = new IntSerializer(ioc); - ioc.longSerializer = new LongSerializer(ioc); - ioc.stringSerializer = new StringSerializer(ioc, ioc.DataType.STRING); - ioc.dateTimeSerializer = new DateTimeSerializer(ioc); - ioc.doubleSerializer = new DoubleSerializer(ioc); - ioc.floatSerializer = new FloatSerializer(ioc); - ioc.listSerializer = new ArraySerializer(ioc, ioc.DataType.LIST); - ioc.mapSerializer = new MapSerializer(ioc); - ioc.setSerializer = new SetSerializer(ioc, ioc.DataType.SET); - ioc.uuidSerializer = new UuidSerializer(ioc); - ioc.edgeSerializer = new EdgeSerializer(ioc); - ioc.pathSerializer = new PathSerializer(ioc); - ioc.propertySerializer = new PropertySerializer(ioc); - ioc.vertexSerializer = new VertexSerializer(ioc); - ioc.vertexPropertySerializer = new VertexPropertySerializer(ioc); - ioc.bigIntegerSerializer = new BigIntegerSerializer(ioc); - ioc.byteSerializer = new ByteSerializer(ioc); - ioc.binarySerializer = new BinarySerializer(ioc); - ioc.shortSerializer = new ShortSerializer(ioc); - ioc.booleanSerializer = new BooleanSerializer(ioc); - ioc.markerSerializer = new MarkerSerializer(ioc); - ioc.unspecifiedNullSerializer = new UnspecifiedNullSerializer(ioc); - ioc.enumSerializer = new EnumSerializer(ioc); - - // Register stub serializers for unimplemented v4 types - new StubSerializer(ioc, ioc.DataType.TREE, 'Tree'); - new StubSerializer(ioc, ioc.DataType.GRAPH, 'Graph'); - new StubSerializer(ioc, ioc.DataType.COMPOSITEPDT, 'CompositePDT'); - new StubSerializer(ioc, ioc.DataType.PRIMITIVEPDT, 'PrimitivePDT'); +ioc.intSerializer = new IntSerializer(ioc); +ioc.longSerializer = new LongSerializer(ioc); +ioc.stringSerializer = new StringSerializer(ioc, ioc.DataType.STRING); +ioc.dateTimeSerializer = new DateTimeSerializer(ioc); +ioc.doubleSerializer = new DoubleSerializer(ioc); +ioc.floatSerializer = new FloatSerializer(ioc); +ioc.listSerializer = new ArraySerializer(ioc, ioc.DataType.LIST); +ioc.mapSerializer = new MapSerializer(ioc); +ioc.setSerializer = new SetSerializer(ioc, ioc.DataType.SET); +ioc.uuidSerializer = new UuidSerializer(ioc); +ioc.edgeSerializer = new EdgeSerializer(ioc); +ioc.pathSerializer = new PathSerializer(ioc); +ioc.propertySerializer = new PropertySerializer(ioc); +ioc.vertexSerializer = new VertexSerializer(ioc); +ioc.vertexPropertySerializer = new VertexPropertySerializer(ioc); +ioc.graphSerializer = new GraphSerializer(ioc); +ioc.bigIntegerSerializer = new BigIntegerSerializer(ioc); +ioc.byteSerializer = new ByteSerializer(ioc); +ioc.binarySerializer = new BinarySerializer(ioc); +ioc.shortSerializer = new ShortSerializer(ioc); +ioc.booleanSerializer = new BooleanSerializer(ioc); +ioc.markerSerializer = new MarkerSerializer(ioc); +ioc.unspecifiedNullSerializer = new UnspecifiedNullSerializer(ioc); +ioc.enumSerializer = new EnumSerializer(ioc); + +// Register stub serializers for unimplemented v4 types +new StubSerializer(ioc, ioc.DataType.TREE, 'Tree'); +new StubSerializer(ioc, ioc.DataType.COMPOSITEPDT, 'CompositePDT'); +new StubSerializer(ioc, ioc.DataType.PRIMITIVEPDT, 'PrimitivePDT'); ioc.numberSerializationStrategy = new NumberSerializationStrategy(ioc); ioc.anySerializer = new AnySerializer(ioc, anySerializerOptions); @@ -163,6 +164,7 @@ export const { propertySerializer, vertexSerializer, vertexPropertySerializer, + graphSerializer, bigIntegerSerializer, byteSerializer, binarySerializer, diff --git a/gremlin-js/gremlin-javascript/lib/structure/io/binary/internals/AnySerializer.js b/gremlin-js/gremlin-javascript/lib/structure/io/binary/internals/AnySerializer.js index bfc20c00162..bcf78436f59 100644 --- a/gremlin-js/gremlin-javascript/lib/structure/io/binary/internals/AnySerializer.js +++ b/gremlin-js/gremlin-javascript/lib/structure/io/binary/internals/AnySerializer.js @@ -40,6 +40,7 @@ export default class AnySerializer { ioc.propertySerializer, ioc.vertexSerializer, ioc.vertexPropertySerializer, + ioc.graphSerializer, ioc.enumSerializer, ioc.stringSerializer, ioc.binarySerializer, diff --git a/gremlin-js/gremlin-javascript/lib/structure/io/binary/internals/GraphSerializer.js b/gremlin-js/gremlin-javascript/lib/structure/io/binary/internals/GraphSerializer.js new file mode 100644 index 00000000000..abf8ac5e2e8 --- /dev/null +++ b/gremlin-js/gremlin-javascript/lib/structure/io/binary/internals/GraphSerializer.js @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { Buffer } from 'buffer'; +import { Graph, Vertex, Edge, VertexProperty } from '../../../graph.js'; + +export default class GraphSerializer { + constructor(ioc) { + this.ioc = ioc; + this.ioc.serializers[ioc.DataType.GRAPH] = this; + } + + canBeUsedFor(value) { + return value instanceof Graph; + } + + serialize(item, fullyQualifiedFormat = true) { + if (item === undefined || item === null) { + if (fullyQualifiedFormat) { + return Buffer.from([this.ioc.DataType.GRAPH, 0x01]); + } + // value-only null fallback: zero vertices + zero edges + const zeroInt = [0x00, 0x00, 0x00, 0x00]; + return Buffer.from([...zeroInt, ...zeroInt]); + } + + const bufs = []; + if (fullyQualifiedFormat) { + bufs.push(Buffer.from([this.ioc.DataType.GRAPH, 0x00])); + } + + const vertices = item.vertices ? Array.from(item.vertices.values()) : []; + const edges = item.edges ? Array.from(item.edges.values()) : []; + + // {vertex_count} + bufs.push(this.ioc.intSerializer.serialize(vertices.length, false)); + + // vertices + for (const v of vertices) { + // {id} + bufs.push(this.ioc.anySerializer.serialize(v.id)); + + // {label} as 1-element list (value-only) + bufs.push(this.ioc.listSerializer.serialize([v.label], false)); + + const vps = Array.isArray(v.properties) ? v.properties : []; + + // {vp_count} + bufs.push(this.ioc.intSerializer.serialize(vps.length, false)); + + for (const vp of vps) { + // {vp_id} + bufs.push(this.ioc.anySerializer.serialize(vp.id)); + + // {vp_label} as 1-element list (value-only) + bufs.push(this.ioc.listSerializer.serialize([vp.label], false)); + + // {vp_value} + bufs.push(this.ioc.anySerializer.serialize(vp.value)); + + // {parent} (always null) + bufs.push(this.ioc.unspecifiedNullSerializer.serialize(null)); + + // {meta_props} as value-only List + const metaProps = Array.isArray(vp.properties) ? vp.properties : []; + bufs.push(this.ioc.listSerializer.serialize(metaProps, false)); + } + } + + // {edge_count} + bufs.push(this.ioc.intSerializer.serialize(edges.length, false)); + + // edges + for (const e of edges) { + // {id} + bufs.push(this.ioc.anySerializer.serialize(e.id)); + + // {label} as 1-element list (value-only) + bufs.push(this.ioc.listSerializer.serialize([e.label], false)); + + // {inV_id} + bufs.push(this.ioc.anySerializer.serialize(e.inV && e.inV.id)); + + // {inV_label} (always null placeholder, fully-qualified) + bufs.push(this.ioc.unspecifiedNullSerializer.serialize(null)); + + // {outV_id} + bufs.push(this.ioc.anySerializer.serialize(e.outV && e.outV.id)); + + // {outV_label} (always null placeholder, fully-qualified) + bufs.push(this.ioc.unspecifiedNullSerializer.serialize(null)); + + // {parent} (always null) + bufs.push(this.ioc.unspecifiedNullSerializer.serialize(null)); + + // {edge_props} as value-only List + const props = Array.isArray(e.properties) ? e.properties : []; + bufs.push(this.ioc.listSerializer.serialize(props, false)); + } + + return Buffer.concat(bufs); + } + + /** + * Async deserialization of graph value bytes from a StreamReader. + * @param {StreamReader} reader + * @param {number} valueFlag + * @param {number} typeCode + * @returns {Promise} + */ + async deserializeValue(reader, valueFlag, typeCode) { + const graph = new Graph(); + + // {vertex_count} bare int + const vertexCount = await this.ioc.intSerializer.deserializeBare(reader); + for (let i = 0; i < vertexCount; i++) { + // {id} fully qualified + const vId = await this.ioc.anySerializer.deserialize(reader); + + // {label} value-only list, first element + const vLabelList = await this.ioc.listSerializer.deserializeValue(reader, 0x00, this.ioc.DataType.LIST); + const vLabel = Array.isArray(vLabelList) && vLabelList.length > 0 ? vLabelList[0] : vLabelList; + + const vertex = new Vertex(vId, vLabel, []); + graph.vertices.set(vId, vertex); + + // {vp_count} bare int + const vpCount = await this.ioc.intSerializer.deserializeBare(reader); + for (let j = 0; j < vpCount; j++) { + // {vp_id} fully qualified + const vpId = await this.ioc.anySerializer.deserialize(reader); + + // {vp_label} value-only list, first element + const vpLabelList = await this.ioc.listSerializer.deserializeValue(reader, 0x00, this.ioc.DataType.LIST); + const vpLabel = Array.isArray(vpLabelList) && vpLabelList.length > 0 ? vpLabelList[0] : vpLabelList; + + // {vp_value} fully qualified + const vpValue = await this.ioc.anySerializer.deserialize(reader); + + // {parent} fully qualified (always null) + await this.ioc.anySerializer.deserialize(reader); + + // {meta_props} value-only list + const metaProps = await this.ioc.listSerializer.deserializeValue(reader, 0x00, this.ioc.DataType.LIST); + + const vp = new VertexProperty(vpId, vpLabel, vpValue, metaProps || []); + vertex.properties.push(vp); + } + } + + // {edge_count} bare int + const edgeCount = await this.ioc.intSerializer.deserializeBare(reader); + for (let i = 0; i < edgeCount; i++) { + // {id} fully qualified + const eId = await this.ioc.anySerializer.deserialize(reader); + + // {label} value-only list, first element + const eLabelList = await this.ioc.listSerializer.deserializeValue(reader, 0x00, this.ioc.DataType.LIST); + const eLabel = Array.isArray(eLabelList) && eLabelList.length > 0 ? eLabelList[0] : eLabelList; + + // {inV_id} fully qualified + const inVId = await this.ioc.anySerializer.deserialize(reader); + + // {inV_label} fully qualified (always null placeholder) — discard + await this.ioc.anySerializer.deserialize(reader); + + // {outV_id} fully qualified + const outVId = await this.ioc.anySerializer.deserialize(reader); + + // {outV_label} fully qualified (always null placeholder) — discard + await this.ioc.anySerializer.deserialize(reader); + + // {parent} fully qualified (always null) — discard + await this.ioc.anySerializer.deserialize(reader); + + // {edge_props} value-only list + const edgeProps = await this.ioc.listSerializer.deserializeValue(reader, 0x00, this.ioc.DataType.LIST); + + // Reuse vertex instances already in graph.vertices, otherwise build stand-ins + const inV = graph.vertices.get(inVId) || new Vertex(inVId, '', []); + const outV = graph.vertices.get(outVId) || new Vertex(outVId, '', []); + + const edge = new Edge(eId, outV, eLabel, inV, edgeProps || []); + graph.edges.set(eId, edge); + } + + return graph; + } + + /** + * Async fully-qualified deserialization from a StreamReader. + * @param {StreamReader} reader + * @returns {Promise} + */ + async deserialize(reader) { + const type_code = await reader.readUInt8(); + if (type_code !== this.ioc.DataType.GRAPH) { + throw new Error(`GraphSerializer: unexpected {type_code}=0x${type_code.toString(16)}`); + } + const value_flag = await reader.readUInt8(); + if (value_flag === 0x01) { + return null; + } + if (value_flag !== 0x00) { + throw new Error(`GraphSerializer: unexpected {value_flag}=0x${value_flag.toString(16)}`); + } + return this.deserializeValue(reader, value_flag, type_code); + } +} diff --git a/gremlin-js/gremlin-javascript/test/cucumber/feature-steps.js b/gremlin-js/gremlin-javascript/test/cucumber/feature-steps.js index 08830a407e6..bb956a2538d 100644 --- a/gremlin-js/gremlin-javascript/test/cucumber/feature-steps.js +++ b/gremlin-js/gremlin-javascript/test/cucumber/feature-steps.js @@ -29,7 +29,7 @@ import { use, expect } from 'chai'; use(chaiString); import { inspect, format, inherits } from 'util'; import { gremlin } from './gremlin.js'; -import { Path, Vertex, Edge, Property } from '../../lib/structure/graph.js'; +import { Path, Vertex, Edge, Property, Graph } from '../../lib/structure/graph.js'; import { statics } from '../../lib/process/graph-traversal.js'; import { t, P, direction, merge, barrier, cardinality, column, order, TextP, IO, pick, pop, scope, operator, withOptions } from '../../lib/process/traversal.js'; import { toLong } from '../../lib/utils.js'; @@ -73,18 +73,12 @@ const ignoreReason = { nullKeysInMapNotSupportedWell: "Javascript does not nicely support 'null' as a key in Map instances", floatingPointIssues: "Javascript floating point numbers not working in this case", uuidSerializationIssues: "Javascript does not serialize to a UUID object, which complicates test assertions", - subgraphStepNotSupported: "Javascript does not yet support subgraph()", treeStepNotSupported: "Javascript does not yet support tree()", needsFurtherInvestigation: '', }; // An associative array for ignored feature tests containing the scenario name as key const ignoredScenarios = { - // javascript doesn't have subgraph() step yet - 'g_VX1X_outEXknowsX_subgraphXsgX_name_capXsgX': new IgnoreError(ignoreReason.subgraphStepNotSupported), - 'g_V_repeatXbothEXcreatedX_subgraphXsgX_outVX_timesX5X_name_dedup_capXsgX': new IgnoreError(ignoreReason.subgraphStepNotSupported), - 'g_V_outEXnoexistX_subgraphXsgXcapXsgX': new IgnoreError(ignoreReason.subgraphStepNotSupported), - 'g_E_hasXweight_0_5X_subgraphXaX_selectXaX': new IgnoreError(ignoreReason.subgraphStepNotSupported), // javascript doesn't have tree() step yet 'g_VX1X_out_out_tree_byXnameX': new IgnoreError(ignoreReason.treeStepNotSupported), 'g_VX1X_out_out_tree': new IgnoreError(ignoreReason.treeStepNotSupported), @@ -257,8 +251,50 @@ Then(/^the result should be (\w+)$/, function assertResult(characterizedAs, resu } }); -Then('the result should be a subgraph with the following', _ => { - // subgraph is not supported yet in javascript +Then('the result should be a subgraph with the following', function (resultTable) { + if (this.result instanceof Error) { + console.error('Error encountered:', this.result.message, this.result.stack); + } + expect(this.result).to.not.be.a.instanceof(Error); + + // 'iterated next' surfaces the Graph directly; 'iterated to list' wraps it in an array. + const sg = Array.isArray(this.result) ? this.result[0] : this.result; + expect(sg).to.be.an.instanceof(Graph); + + // No data table means there's nothing further to assert. + if (!resultTable || typeof resultTable.raw !== 'function') { + return; + } + const raw = resultTable.raw(); + if (!raw || raw.length === 0) { + return; + } + const header = raw[0][0]; + const assertingVertices = header === 'vertices'; + + const dataRows = typeof resultTable.rows === 'function' ? resultTable.rows() : raw.slice(1); + + if (assertingVertices) { + const expectedVertices = dataRows.map(row => parseValue.call(this, row[0])); + expect(sg.vertices.size).to.equal(expectedVertices.length); + for (const expected of expectedVertices) { + expect(sg.vertices.has(expected.id)).to.equal(true, + format('subgraph is missing vertex %s', inspect(expected.id))); + const actual = sg.vertices.get(expected.id); + expect(actual.label).to.equal(expected.label); + } + } else { + const expectedEdges = dataRows.map(row => parseValue.call(this, row[0])); + expect(sg.edges.size).to.equal(expectedEdges.length); + for (const expected of expectedEdges) { + expect(sg.edges.has(expected.id)).to.equal(true, + format('subgraph is missing edge %s', inspect(expected.id))); + const actual = sg.edges.get(expected.id); + expect(actual.label).to.equal(expected.label); + expect(actual.outV.id).to.equal(expected.outV.id); + expect(actual.inV.id).to.equal(expected.inV.id); + } + } }); Then('the result should be a tree with a structure of', _ => { diff --git a/gremlin-js/gremlin-javascript/test/cucumber/gremlin.js b/gremlin-js/gremlin-javascript/test/cucumber/gremlin.js index dccd1a3cf34..aef279b28a8 100644 --- a/gremlin-js/gremlin-javascript/test/cucumber/gremlin.js +++ b/gremlin-js/gremlin-javascript/test/cucumber/gremlin.js @@ -1022,6 +1022,7 @@ const gremlins = { g_V_hasLabelXpersonX_valuesXageX_asString: [function({g}) { return g.V().hasLabel("person").values("age").asString() }], g_V_hasLabelXpersonX_valuesXageX_order_fold_asStringXlocalX: [function({g}) { return g.V().hasLabel("person").values("age").order().fold().asString(Scope.local) }], g_V_hasLabelXpersonX_valuesXageX_asString_concatX_years_oldX: [function({g}) { return g.V().hasLabel("person").values("age").asString().concat(" years old") }], + g_V_outEXknowsX_subgraphXsgX_capXsgX_asString: [function({g}) { return g.V().outE("knows").subgraph("sg").cap("sg").asString() }], g_call: [function({g}) { return g.call() }], g_callXlistX: [function({g}) { return g.call("--list") }], g_callXlistX_withXstring_stringX: [function({g}) { return g.call("--list").with_("service", "tinker.search") }], diff --git a/gremlin-js/gremlin-javascript/test/unit/graph-serializer-test.js b/gremlin-js/gremlin-javascript/test/unit/graph-serializer-test.js new file mode 100644 index 00000000000..a7297cf995b --- /dev/null +++ b/gremlin-js/gremlin-javascript/test/unit/graph-serializer-test.js @@ -0,0 +1,194 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { assert } from 'chai'; +import { Buffer } from 'buffer'; +import ioc from '../../lib/structure/io/binary/GraphBinary.js'; +import StreamReader from '../../lib/structure/io/binary/internals/StreamReader.js'; +import { Graph, Vertex, Edge, VertexProperty, Property } from '../../lib/structure/graph.js'; + +/** + * Round-trips a value through serialize + StreamReader -> anySerializer.deserialize. + * Mirrors the pattern used in test/unit/graphbinary/async-deserialize-test.js. + */ +async function roundTripAny(value) { + const buf = ioc.anySerializer.serialize(value); + const reader = StreamReader.fromBuffer(buf); + return ioc.anySerializer.deserialize(reader); +} + +describe('Graph', () => { + describe('toString()', () => { + it('renders an empty graph with zero counts', () => { + const graph = new Graph(); + assert.equal(graph.toString(), 'graph[vertices:0 edges:0]'); + }); + + it('renders the counts of vertices and edges', () => { + const graph = new Graph(); + const v1 = new Vertex(1, 'person', []); + const v2 = new Vertex(2, 'person', []); + graph.vertices.set(1, v1); + graph.vertices.set(2, v2); + graph.edges.set(3, new Edge(3, v1, 'knows', v2, [])); + assert.equal(graph.toString(), 'graph[vertices:2 edges:1]'); + }); + }); +}); + +describe('GraphSerializer', () => { + describe('round-trip', () => { + it('preserves vertices, edges, vertex-properties, and properties', async () => { + const graph = new Graph(); + + const v1 = new Vertex(1, 'person', []); + const v2 = new Vertex(2, 'person', []); + graph.vertices.set(1, v1); + graph.vertices.set(2, v2); + + // VertexProperty on v1 with one meta-property + const vp1 = new VertexProperty(4, 'name', 'marko', [new Property('acl', 'public')]); + v1.properties.push(vp1); + + // Edge v1 -knows-> v2 with weight property + const e1 = new Edge(3, v1, 'knows', v2, [new Property('weight', 0.5)]); + graph.edges.set(3, e1); + + const output = await roundTripAny(graph); + + assert.instanceOf(output, Graph); + assert.equal(output.vertices.size, 2); + assert.equal(output.edges.size, 1); + + const rv1 = output.vertices.get(1); + assert.instanceOf(rv1, Vertex); + assert.equal(rv1.label, 'person'); + assert.equal(rv1.properties.length, 1); + + const rvp1 = rv1.properties[0]; + assert.instanceOf(rvp1, VertexProperty); + assert.equal(rvp1.value, 'marko'); + assert.equal(rvp1.label, 'name'); + assert.equal(rvp1.properties.length, 1); + assert.equal(rvp1.properties[0].key, 'acl'); + assert.equal(rvp1.properties[0].value, 'public'); + + const rv2 = output.vertices.get(2); + assert.instanceOf(rv2, Vertex); + assert.equal(rv2.label, 'person'); + + const re1 = output.edges.get(3); + assert.instanceOf(re1, Edge); + assert.equal(re1.label, 'knows'); + assert.equal(re1.outV.id, 1); + assert.equal(re1.inV.id, 2); + assert.equal(re1.properties.length, 1); + assert.equal(re1.properties[0].key, 'weight'); + assert.equal(re1.properties[0].value, 0.5); + }); + + it('handles an empty graph', async () => { + const graph = new Graph(); + const output = await roundTripAny(graph); + + assert.instanceOf(output, Graph); + assert.equal(output.vertices.size, 0); + assert.equal(output.edges.size, 0); + }); + + it('handles vertices with no properties and edges with no properties', async () => { + const graph = new Graph(); + const v1 = new Vertex(10, 'person', []); + const v2 = new Vertex(20, 'software', []); + graph.vertices.set(10, v1); + graph.vertices.set(20, v2); + graph.edges.set(30, new Edge(30, v1, 'created', v2, [])); + + const output = await roundTripAny(graph); + assert.instanceOf(output, Graph); + assert.equal(output.vertices.size, 2); + assert.equal(output.edges.size, 1); + + const re = output.edges.get(30); + assert.equal(re.label, 'created'); + assert.equal(re.outV.id, 10); + assert.equal(re.inV.id, 20); + // Edge points at the same Vertex instance we already deserialized into the graph + assert.strictEqual(re.outV, output.vertices.get(10)); + assert.strictEqual(re.inV, output.vertices.get(20)); + assert.equal(re.properties.length, 0); + }); + + it('handles string ids', async () => { + const graph = new Graph(); + const v1 = new Vertex('a', 'person', []); + const v2 = new Vertex('b', 'person', []); + graph.vertices.set('a', v1); + graph.vertices.set('b', v2); + graph.edges.set('e1', new Edge('e1', v1, 'knows', v2, [])); + + const output = await roundTripAny(graph); + assert.equal(output.vertices.size, 2); + assert.equal(output.edges.size, 1); + assert.equal(output.vertices.get('a').label, 'person'); + assert.equal(output.edges.get('e1').outV.id, 'a'); + assert.equal(output.edges.get('e1').inV.id, 'b'); + }); + }); + + describe('null handling', () => { + it('serializes null as fully-qualified null bytes', () => { + const buf = ioc.graphSerializer.serialize(null, true); + assert.deepEqual([...buf], [ioc.DataType.GRAPH, 0x01]); + }); + + it('deserialize() returns null for value_flag=0x01', async () => { + const buf = Buffer.from([ioc.DataType.GRAPH, 0x01]); + const reader = StreamReader.fromBuffer(buf); + const result = await ioc.graphSerializer.deserialize(reader); + assert.isNull(result); + }); + + it('rejects an unexpected type_code', async () => { + const buf = Buffer.from([ioc.DataType.VERTEX, 0x00]); + const reader = StreamReader.fromBuffer(buf); + try { + await ioc.graphSerializer.deserialize(reader); + assert.fail('should have thrown'); + } catch (err) { + assert.match(err.message, /unexpected \{type_code\}/); + } + }); + }); + + describe('AnySerializer routing', () => { + it('selects GraphSerializer for a Graph instance', () => { + const graph = new Graph(); + const serializer = ioc.anySerializer.getSerializerCanBeUsedFor(graph); + assert.strictEqual(serializer, ioc.graphSerializer); + }); + + it('serializes a Graph via anySerializer with the GRAPH type code', () => { + const graph = new Graph(); + const buf = ioc.anySerializer.serialize(graph); + assert.equal(buf[0], ioc.DataType.GRAPH); + assert.equal(buf[1], 0x00); + }); + }); +}); diff --git a/gremlin-python/src/main/python/gremlin_python/structure/graph.py b/gremlin-python/src/main/python/gremlin_python/structure/graph.py index 2ff6e56174a..61f92c30654 100644 --- a/gremlin-python/src/main/python/gremlin_python/structure/graph.py +++ b/gremlin-python/src/main/python/gremlin_python/structure/graph.py @@ -26,7 +26,7 @@ def __init__(self): self.edges = {} def __repr__(self): - return "graph[]" + return "graph[vertices:" + str(len(self.vertices)) + " edges:" + str(len(self.edges)) + "]" class Element(object): diff --git a/gremlin-python/src/main/python/tests/feature/gremlin.py b/gremlin-python/src/main/python/tests/feature/gremlin.py index 9cd84e34ff4..8dd5394e02f 100644 --- a/gremlin-python/src/main/python/tests/feature/gremlin.py +++ b/gremlin-python/src/main/python/tests/feature/gremlin.py @@ -996,6 +996,7 @@ 'g_V_hasLabelXpersonX_valuesXageX_asString': [(lambda g:g.V().has_label('person').values('age').as_string())], 'g_V_hasLabelXpersonX_valuesXageX_order_fold_asStringXlocalX': [(lambda g:g.V().has_label('person').values('age').order().fold().as_string(Scope.local))], 'g_V_hasLabelXpersonX_valuesXageX_asString_concatX_years_oldX': [(lambda g:g.V().has_label('person').values('age').as_string().concat(' years old'))], + 'g_V_outEXknowsX_subgraphXsgX_capXsgX_asString': [(lambda g:g.V().out_e('knows').subgraph('sg').cap('sg').as_string())], 'g_call': [(lambda g:g.call())], 'g_callXlistX': [(lambda g:g.call('--list'))], 'g_callXlistX_withXstring_stringX': [(lambda g:g.call('--list').with_('service', 'tinker.search'))], diff --git a/gremlin-python/src/main/python/tests/unit/structure/test_graph.py b/gremlin-python/src/main/python/tests/unit/structure/test_graph.py index 21d267ea517..6b1f7ef8bf1 100644 --- a/gremlin-python/src/main/python/tests/unit/structure/test_graph.py +++ b/gremlin-python/src/main/python/tests/unit/structure/test_graph.py @@ -21,6 +21,7 @@ from gremlin_python.statics import long from gremlin_python.structure.graph import Edge +from gremlin_python.structure.graph import Graph from gremlin_python.structure.graph import Property from gremlin_python.structure.graph import Vertex from gremlin_python.structure.graph import VertexProperty @@ -92,6 +93,19 @@ def test_graph_objects(self): assert i == j assert i.__hash__() == hash(i) + def test_graph_repr(self): + # empty graph + g = Graph() + assert "graph[vertices:0 edges:0]" == repr(g) + + # graph with two vertices and one edge + v1 = Vertex(1, "person") + v2 = Vertex(2, "person") + g.vertices[1] = v1 + g.vertices[2] = v2 + g.edges[3] = Edge(3, v1, "knows", v2) + assert "graph[vertices:2 edges:1]" == repr(g) + def test_path(self): path = Path([set(["a", "b"]), set(["c", "b"]), set([])], [1, Vertex(1), "hello"]) assert "path[1, v[1], hello]" == str(path) diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/language/translator/translations.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/language/translator/translations.json index 44163ea0f74..3c9c17c495a 100644 --- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/language/translator/translations.json +++ b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/language/translator/translations.json @@ -20419,6 +20419,23 @@ } ] }, + { + "scenario": "g_V_outEXknowsX_subgraphXsgX_capXsgX_asString", + "traversals": [ + { + "original": "g.V().outE(\"knows\").subgraph(\"sg\").cap(\"sg\").asString()", + "language": "g.V().outE(\"knows\").subgraph(\"sg\").cap(\"sg\").asString()", + "canonical": "g.V().outE(\"knows\").subgraph(\"sg\").cap(\"sg\").asString()", + "anonymized": "g.V().outE(string0).subgraph(string1).cap(string1).asString()", + "dotnet": "g.V().OutE(\"knows\").Subgraph(\"sg\").Cap(\"sg\").AsString()", + "go": "g.V().OutE(\"knows\").Subgraph(\"sg\").Cap(\"sg\").AsString()", + "groovy": "g.V().outE(\"knows\").subgraph(\"sg\").cap(\"sg\").asString()", + "java": "g.V().outE(\"knows\").subgraph(\"sg\").cap(\"sg\").asString()", + "javascript": "g.V().outE(\"knows\").subgraph(\"sg\").cap(\"sg\").asString()", + "python": "g.V().out_e('knows').subgraph('sg').cap('sg').as_string()" + } + ] + }, { "scenario": "g_call", "traversals": [ diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features/map/AsString.feature b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features/map/AsString.feature index 05abdd6448e..9f2ff22090f 100644 --- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features/map/AsString.feature +++ b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features/map/AsString.feature @@ -209,4 +209,16 @@ Feature: Step - asString() | 29 years old | | 27 years old | | 32 years old | - | 35 years old | \ No newline at end of file + | 35 years old | + + @StepSubgraph + Scenario: g_V_outEXknowsX_subgraphXsgX_capXsgX_asString + Given the modern graph + And the traversal of + """ + g.V().outE("knows").subgraph("sg").cap("sg").asString() + """ + When iterated to list + Then the result should be unordered + | result | + | tinkergraph[vertices:3 edges:2] |