Generic Data Structures #33

elazarg · 2024-04-14T17:55:40Z

Decompose the data structures into domain-specific and generic data structures.

The ds package has zero dependencies, and it knows nothing about numbers or network.

Interfaces (all in interfaces.go):

Comparable (Equal, Copy)
Hashable (Comparable, Hash)
Set (Hashable + set operations)
Product of A x B (partitions, left and right projections, swap)
TripleSet, convenience (and associativity agnostic) interface for A x B x C

Implementations:

HashMap - a simple adapter around a go map, allowing Hashable keys.
HashSet
MultiMap is a non-injective map. It gives a simple "inverse" on a HashMap.
ProductLeft ~~and ProductRight~~ is a map-based implementation of a cartesian product; it implements Product as a one-to-one mapping from sets to sets, where each key-value pairs encodes the cross product of its two items.
TripleSetLeft and TripleSetRight and TripleSetOuter are right-, left- and outer-associative implementations of TripleSet.
DisjointSum is tagged-union of two sets. Go's generics place some limitations on the implementation; The empty element must always be explicitly passed to the constructor.
~~UnitSet is an idea - using sets of size 1 as the identity element of Product. It's probably not really helpful.~~

The interval package has interval and intervalset. Nothing particularly new about them.

netp is unchanged.

The netset package is where handles sets of network elements.

ICMPSet is largely the one proposed in Add ICMP set #23.
IPBlock is adapted to implement Set.
TCPUDPSet is a TripleSet of ProtocolSet x PortSet x PortSet
TransportSet is a disjoint union of TCPUDPSet + ICMPSet
ConnectionSet is a TripleSet of IPBlock x IPBlock x TransportSet

The package connection now contains the state-aware connectionset (named Set), and its json-formatting. In my opinion it should be moved to the analyzer.

Many tests are adapted. ~~but some data structures do not have dedicated tests.~~

There is no implementation of a dynamically-bounded tuple, since I could not find the proper way to fit that into a generic datastructure with a well defined degenerate edge case.

The diff is probably not useful; it should be easier to read the code.

adisos

Looks good, added few initial comments.

pkg/netset/connectionset.go

pkg/netset/tcpudpset.go

pkg/ds/interfaces.go

pkg/ds/hashmap.go

pkg/netset/connectionset.go

elazarg · 2024-06-19T19:23:59Z

We may need to use ordered maps instead of hash maps, so the lists of partitions are consistently ordered.

adisos

few more questions and comments

pkg/ds/interfaces.go

pkg/netset/icmpset.go

pkg/netset/connectionset.go

pkg/ds/unitset.go

pkg/ds/tripleset_outer.go

pkg/netp/icmp.go

Signed-off-by: Elazar Gershuni <[email protected]>

Also, rename NewMap to NewHashMap Signed-off-by: Elazar Gershuni <[email protected]>

Signed-off-by: Elazar Gershuni <[email protected]>

elazarg · 2024-06-25T18:14:58Z

Added an elaborated README, and changed implementation of MultiMap to use HashSet.

elazarg · 2024-06-26T19:24:15Z

BTW, I think the class ICMP is better named ICMPRule, and is a set. The file icmp.go in the netp package should only have functions that are related to ICMP proper. Such change would also solve the need to expose implementation details of the ICMP class to the ICMPSet class.

elazarg · 2024-07-03T15:44:54Z

@adisos is there anything left for me to implement/document so this PR can be merged?

adisos

few small comments and questions

pkg/ds/hashmap.go

adisos · 2024-07-08T07:05:37Z

pkg/ds/product_left.go

+	return m.m.IsEmpty()
+}
+
+// Size returns the number of unique pairs in the Product object


the Size documentation is incorrect? it does not return the number of pairs..

adisos · 2024-07-08T07:06:17Z

pkg/ds/interfaces.go

+// Product is a cartesian product of sets S1 x S2
+type Product[S1 Set[S1], S2 Set[S2]] interface {
+	Set[Product[S1, S2]]
+	Partitions() []Pair[S1, S2]


Suggested change

Partitions() []Pair[S1, S2]

// Partitions returns a slice of pairs in the product set (note that the order is arbitrary)

Partitions() []Pair[S1, S2]

adisos · 2024-07-08T07:07:27Z

pkg/ds/interfaces.go

+type Product[S1 Set[S1], S2 Set[S2]] interface {
+	Set[Product[S1, S2]]
+	Partitions() []Pair[S1, S2]
+	NumPartitions() int


should explain the difference between NumPartitions and Size ?

adisos · 2024-07-08T07:08:02Z

pkg/ds/interfaces.go

+	Set[Product[S1, S2]]
+	Partitions() []Pair[S1, S2]
+	NumPartitions() int
+	Left(empty S1) S1


Suggested change

Left(empty S1) S1

// Left returns the left projection from pairs in the product set on S1, given input of an empty set in S1

Left(empty S1) S1

adisos · 2024-07-08T07:08:27Z

pkg/ds/interfaces.go

+	Partitions() []Pair[S1, S2]
+	NumPartitions() int
+	Left(empty S1) S1
+	Right(empty S2) S2


Suggested change

Right(empty S2) S2

// Right returns the right projection from pairs in the product set S2, given input of an empty set in S2

Right(empty S2) S2

adisos · 2024-07-08T07:08:48Z

pkg/ds/interfaces.go

+	NumPartitions() int
+	Left(empty S1) S1
+	Right(empty S2) S2
+	Swap() Product[S2, S1]


Suggested change

Swap() Product[S2, S1]

// Swap returns a new Product object, built from the input object, with left and right swapped

Swap() Product[S2, S1]

adisos · 2024-07-08T07:09:58Z

@adisos is there anything left for me to implement/document so this PR can be merged?

added few small comments, still reviewing the tests and some implementation details.

Co-authored-by: Adi Sosnovich <[email protected]>

Signed-off-by: Elazar Gershuni <[email protected]>

interval: * Separate tests from interval_test.go * Improve documentation. * Export and set-like functions that are well defined. * Rename interval.Subtract to interval.SubtractSplit, and add tests. * Handle empty cases first. * Preallocate Elements. intervalset: * Guard Size() from overflow, and use intervalset.CalculateSize(). * Handle empty cases first. * Remove String() method, since it is not obvious; clients should implement. Signed-off-by: Elazar Gershuni <[email protected]>

elazarg · 2024-07-08T15:12:05Z

I've improved and fixed the documentation of interface.go. @adisos please re-review this part.

Additionally, I've improved the interval package.

interval:

Separate tests from intervalset_test.go
Improve documentation.
Export set-like functions that are well defined.
Rename interval.Subtract to interval.SubtractSplit, and add tests.
Handle empty cases first.
Preallocate Elements.

intervalset:

Guard Size() from overflow, and use intervalset.CalculateSize().
Handle empty cases first.
Remove String() method, since it is not obvious; clients should implement.

I think it's better to make GenericInterval and GenericCanonicalSet as generic types (with integral constraints), so clients can choose whether or not to rely on things like Size() being 64bit integer. interval.Interval should be an alias to GenericInterval[int], and interval.Set should be an alias to GenericCanonicalSet[int].

pull-request-size bot added the size/XXL label Apr 14, 2024

elazarg requested a review from zivnevo April 15, 2024 07:08

zivnevo requested a review from adisos April 22, 2024 07:32

adisos mentioned this pull request May 2, 2024

integrate generic data structures #42

Open

adisos reviewed May 2, 2024

View reviewed changes

elazarg requested a review from adisos May 7, 2024 17:07

elazarg force-pushed the generic branch 2 times, most recently from 09bcd6a to 50c0649 Compare May 21, 2024 11:56

elazarg mentioned this pull request May 23, 2024

Generic np-guard/vpc-network-config-analyzer#597

Open

elazarg force-pushed the generic branch 2 times, most recently from d0d92a5 to b0b97bd Compare June 11, 2024 07:58

elazarg force-pushed the generic branch from b0b97bd to 3c3f10a Compare June 18, 2024 21:44

adisos reviewed Jun 25, 2024

View reviewed changes

generic

259c371

Signed-off-by: Elazar Gershuni <[email protected]>

elazarg force-pushed the generic branch from 25210b0 to 259c371 Compare June 25, 2024 16:45

elazarg added 3 commits June 25, 2024 20:24

Update README.md

54d1a32

Signed-off-by: Elazar Gershuni <[email protected]>

Use a HashSet in MultiMap

77f2879

Also, rename NewMap to NewHashMap Signed-off-by: Elazar Gershuni <[email protected]>

add multimap changes

81fbb65

Signed-off-by: Elazar Gershuni <[email protected]>

elazarg mentioned this pull request Jun 26, 2024

Use netp.Protocol np-guard/vpc-network-config-synthesis#153

Open

elazarg requested a review from adisos July 3, 2024 15:44

elazarg mentioned this pull request Jul 6, 2024

Add ICMP set #23

Closed

adisos reviewed Jul 8, 2024

View reviewed changes

elazarg and others added 3 commits July 8, 2024 15:43

Update pkg/ds/hashmap.go

54ec389

Co-authored-by: Adi Sosnovich <[email protected]>

Improve documentation for interfaces.go

82b2ce3

Signed-off-by: Elazar Gershuni <[email protected]>

elazarg requested a review from adisos July 8, 2024 15:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generic Data Structures #33

Generic Data Structures #33

elazarg commented Apr 14, 2024 •

edited

Loading

adisos left a comment

elazarg commented Jun 19, 2024

adisos left a comment

elazarg commented Jun 25, 2024

elazarg commented Jun 26, 2024

elazarg commented Jul 3, 2024

adisos left a comment

adisos Jul 8, 2024

adisos Jul 8, 2024

adisos Jul 8, 2024

adisos Jul 8, 2024

adisos Jul 8, 2024

adisos Jul 8, 2024

adisos commented Jul 8, 2024

elazarg commented Jul 8, 2024 •

edited

Loading

	Partitions() []Pair[S1, S2]
	// Partitions returns a slice of pairs in the product set (note that the order is arbitrary)
	Partitions() []Pair[S1, S2]

	Left(empty S1) S1
	// Left returns the left projection from pairs in the product set on S1, given input of an empty set in S1
	Left(empty S1) S1

	Right(empty S2) S2
	// Right returns the right projection from pairs in the product set S2, given input of an empty set in S2
	Right(empty S2) S2

	Swap() Product[S2, S1]
	// Swap returns a new Product object, built from the input object, with left and right swapped
	Swap() Product[S2, S1]

Generic Data Structures #33

Are you sure you want to change the base?

Generic Data Structures #33

Conversation

elazarg commented Apr 14, 2024 • edited Loading

adisos left a comment

Choose a reason for hiding this comment

elazarg commented Jun 19, 2024

adisos left a comment

Choose a reason for hiding this comment

elazarg commented Jun 25, 2024

elazarg commented Jun 26, 2024

elazarg commented Jul 3, 2024

adisos left a comment

Choose a reason for hiding this comment

adisos Jul 8, 2024

Choose a reason for hiding this comment

adisos Jul 8, 2024

Choose a reason for hiding this comment

adisos Jul 8, 2024

Choose a reason for hiding this comment

adisos Jul 8, 2024

Choose a reason for hiding this comment

adisos Jul 8, 2024

Choose a reason for hiding this comment

adisos Jul 8, 2024

Choose a reason for hiding this comment

adisos commented Jul 8, 2024

elazarg commented Jul 8, 2024 • edited Loading

elazarg commented Apr 14, 2024 •

edited

Loading

elazarg commented Jul 8, 2024 •

edited

Loading