Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query resource monitor #131

Open
wants to merge 14 commits into
base: develop
Choose a base branch
from
Open
37 changes: 37 additions & 0 deletions docs/branch/query-resource-monitor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#Query Resource Monitor
The purpose of this feature is to evaluate and maintain a resource utilization monitor for each
query->cursor execution process.

A query by itself has no resource allocated since it only represents a logical query, once a cursor was created
it enters a state machine with the following stages:

- Not Started - awaiting activation

- Running - Running and fetching data from storage

- Suspended - Suspended and not executing - but scroll resources are still available in store
- Canceled - Suspended and not executing - No scroll resources available in store

A cursor can also be deleted - it will no longer be present or contain any resources.


###Execution
During the cursor execution phase (graph traversal) we will introduce the following capabilities:

- Cancellation
- Resource monitoring

Since each cursor is executed inside the "page" scope - we will maintain these capabilities to be in this resolution.
Meaning we will be able to cancel / pause / resume the execution in the scope of a page size

If the requested pageSize was very large it will be internally partitioned into intermediate size so that we could enforce the
capabilities we presented - approximate max size will be configurable and default to 500



###Resources
A Cursor holds the next resources:

- open cursors (store depended)
- paged data (memory allocated)
- materialized documents (projection index)
1,005 changes: 1,001 additions & 4 deletions docs/tutorial/sample/dragons/csv/Dragons.csv

Large diffs are not rendered by default.

11,507 changes: 11,504 additions & 3 deletions docs/tutorial/sample/dragons/csv/Fire.csv

Large diffs are not rendered by default.

4,910 changes: 4,908 additions & 2 deletions docs/tutorial/sample/dragons/csv/Freeze.csv

Large diffs are not rendered by default.

20 changes: 16 additions & 4 deletions docs/tutorial/sample/dragons/csv/Guilds.csv
Original file line number Diff line number Diff line change
@@ -1,4 +1,16 @@
id,name,description,iconId,url,establishDate
G01-masons,masons,"There were several types of guilds, including the two main categories of merchant guilds and craft guilds[4] but also the frith guild and religious guild.[5] Guilds arose beginning in the High Middle Ages as craftsmen united to protect their common interests. In the German city of Augsburg craft guilds are mentioned in the Towncharter of 1156.[6]",,,30/6/1020
G02-merchant,merchant,"The continental system of guilds and merchants arrived in England after the Norman Conquest, with incorporated societies of merchants in each town or city holding exclusive rights of doing business there. In many cases they became the governing body of a town. For example, London's Guildhall became the seat of the Court of Common Council of the City of London Corporation, the world’s oldest continuously elected local government,[7] whose members to this day must be Freemen of the City.[8] The Freedom of the City, effective from the Middle Ages until 1835, gave the right to trade, and was only bestowed upon members of a Guild or Livery.[9]",,,1/1/520
G03-craft,craft,"In England, specifically in the City of London Corporation, more than 110 guilds,[16] referred to as livery companies, survive today,[17] with the oldest more than a thousand years old.[citation needed] Other groups, such as the Worshipful Company of Tax Advisers, have been formed far more recently. Membership in a livery company is expected for individuals participating in the governance of The City, as the Lord Mayor and the Remembrancer.\n\n\nThe Syndics of the Drapers' Guild by Rembrandt, 1662.\nThe guild system reached a mature state in Germany circa 1300 and held on in German cities into the 19th century, with some special privileges for certain occupations remaining today. In the 15th century, Hamburg had 100 guilds, Cologne 80, and Lübeck 70.[18] The latest guilds to develop in Western Europe were the gremios of Spain: e.g., Valencia (1332) or Toledo (1426).\n\nNot all city economies were controlled by guilds; some cities were free. Where guilds were in control, they shaped labor, production and trade; they had strong controls over instructional capital, and the modern concepts of a lifetime progression of apprentice to craftsman, and then from journeyman eventually to widely recognized master and grandmaster began to emerge. In order to become a master, a journeyman would have to go on a three-year voyage called journeyman years. The practice of the journeyman years still exists in Germany and France.",,,1/2/1330
"id","name","description","iconId","url","establishDate"
"0","Flint of Flint's Finger","Est repellat temporibus sequi est eveniet et aut similique consequuntur.","https://s3.amazonaws.com/uifaces/faces/twitter/teeragit/128.jpg","www.jacklyn-treutel.co","-2112196821935"
"1","Bolling","Excepturi ab officiis sint et est aut ea omnis tenetur fugiat nihil quos.","https://s3.amazonaws.com/uifaces/faces/twitter/laurengray/128.jpg","www.emanuel-bauch.net","-2101529345903"
"2","Graceford of Holyhall","Quo non nesciunt sit enim et sunt ad itaque repellendus illo quos ipsum dicta.","https://s3.amazonaws.com/uifaces/faces/twitter/ryhanhassan/128.jpg","www.ole-greenholt.net","-2067763951979"
"3","Ferren","Qui qui ducimus odio quo est incidunt dolore quae mollitia.","https://s3.amazonaws.com/uifaces/faces/twitter/markjenkins/128.jpg","www.frederick-miller.org","-2131982275053"
"4","Breakstone","Aut quis corrupti praesentium quae sunt veritatis quia ullam qui sit accusantium.","https://s3.amazonaws.com/uifaces/faces/twitter/calebogden/128.jpg","www.hans-vandervort.biz","-2159636258949"
"5","Blackwood of Raventree Hall","Excepturi reprehenderit tenetur velit et asperiores doloremque modi tenetur ullam asperiores.","https://s3.amazonaws.com/uifaces/faces/twitter/vanchesz/128.jpg","www.micah-gibson.org","-2101189511797"
"6","Frey of Riverrun","Sunt est esse tempora in qui laborum consequuntur aut omnis et nobis ex.","https://s3.amazonaws.com/uifaces/faces/twitter/scottkclark/128.jpg","www.regan-dooley.biz","-2126640437716"
"7","Holt","Voluptatum officiis quisquam et et sit laboriosam occaecati quibusdam tenetur non esse error vel.","https://s3.amazonaws.com/uifaces/faces/twitter/ninjad3m0/128.jpg","www.kristoffer-lynch.com","-2162357714360"
"8","Farwynd of the Lonely Light","Possimus sint dolores magnam qui omnis cupiditate dolore error quia enim rem asperiores ut.","https://s3.amazonaws.com/uifaces/faces/twitter/adityasutomo/128.jpg","www.schuyler-carter.info","-2060724243190"
"9","Heddle","Ipsum non laboriosam sapiente animi voluptatem et cum labore quis provident.","https://s3.amazonaws.com/uifaces/faces/twitter/unterdreht/128.jpg","www.lucy-gutmann.com","-2101640132469"
"10","Cave","Necessitatibus deleniti veniam voluptas error quibusdam autem fugiat cupiditate unde commodi deserunt soluta.","https://s3.amazonaws.com/uifaces/faces/twitter/xravil/128.jpg","www.rogers-hammes.org","-2077002313302"
"11","Frey of the Crossing","Officia et modi doloribus illum tempora et veritatis quis rerum.","https://s3.amazonaws.com/uifaces/faces/twitter/larrybolt/128.jpg","www.jayne-cormier.net","-2200183124733"
"12","Flint of Widow's Watch","Delectus corporis alias qui ipsam dolor quod iusto omnis molestiae.","https://s3.amazonaws.com/uifaces/faces/twitter/kaysix_dizzy/128.jpg","www.brendan-mcclure.net","-2059185803101"
"13","Footly of Tumbleton","Ut modi eveniet ut dicta illum omnis magnam sed quae accusamus ipsa.","https://s3.amazonaws.com/uifaces/faces/twitter/smalonso/128.jpg","www.reid-dibbert.io","-2208075327205"
"14","Forrester","Deserunt voluptate enim qui et placeat molestiae aut a corporis deserunt voluptas quia dolores.","https://s3.amazonaws.com/uifaces/faces/twitter/vanchesz/128.jpg","www.valentine-corkery.info","-2154378270177"
4 changes: 0 additions & 4 deletions docs/tutorial/sample/dragons/csv/Horses.csv

This file was deleted.

5 changes: 0 additions & 5 deletions docs/tutorial/sample/dragons/csv/Kingdom.csv

This file was deleted.

4 changes: 0 additions & 4 deletions docs/tutorial/sample/dragons/csv/Knows.csv

This file was deleted.

Loading