Discussion for Documentation

[sehilyi]

The thread of this discussion will cover the questions and comments around the geminid grammar to make documentation.

[wangqianwen0418]

The current documentations [undergoing 📝 ] is available at https://github.com/sehilyi/geminid/wiki/Docs

Some comments after trying the basic:

• 👍 overall, the structure is clear, should be easy for those who have prior experience in declarative visualisation grammar. Not sure about the domain experts, who maybe unfamiliar with visualization grammars
• ❓ the part about track.data and track.metadata is a little bit confusing. Not sure what are the supported data formats
• ❓ how can I rotate a chart, for example, the area chart in basic marks, 90 degree? I have tried to switch the encoding of x and y axis but failed. maybe this is a rare case in genomic visualization?)
• ❓ it is a little bit confusing for me to understand the difference between rect and bar. Also confused about the encoding of width and height of these two marks
• ❓ I didn't see the legend (when set legend: true) of color when "color": {"type": "quantitative", "legend": true}
• 💬 one track has different options based on the value of track.mark, need good documentations to explain what are the unique options of each mark and what are the shared options
• 💬 it makes more sense to me if we move width, height, and static to the layout option
• 💬 As a user, I may want to add a title and some necessary descriptions in the visualization, not just in the code. Maybe it will help if description is an object with options such as title, subtitle, tooltip.

[sehilyi]

• Entire grammar: https://github.com/sehilyi/geminid/blob/master/src/core/geminid.schema.ts
• How each mark is rendered (encoded): https://github.com/sehilyi/geminid/tree/master/src/core/mark

[sehilyi]

For each subsection (e.g., "Point"), we can cover the following contents:

• Short description
• A table (just like the current layout table) (if applicable)
• Example
  • Part of spec (if applicable)
  • Actual example embedded (if applicable)

[wangqianwen0418]

Some questions:

1. is size option also supported when the mark is not point?
2. strokeWidth is currently listed below encoding, is it just a style option, or it can be bound with a data field? The same question goes to opacity.
3. I am little confused about x, xe, and x1e. Would you mind providing more explanations?

[sehilyi]

Thats good question.

1. size is being used for other marks as well (e.g., when size is provided, the width of a bar is encoded with the size, which is defined here).
2. My intension was to support encoding quantitative data with strokeWidth, but did not find a good use case for that. So, currently it does not encode data (this is the part of source code that determines whether to encode data or not, FYI).
3. For most of the cases, x w/ an additional channel, xe, are used ("e" refers to "end"). However, band connection (link) should consider four different channels (start and end positions of two genomic regions), and this is the only case where x1 and x1e are used.

[wangqianwen0418]

Thanks !

1. but size may not work for area and line? Would you list the marks where size encoding is supported?
2. I see the type of opacity is ChannelValue, so it doesn't support data encoding?
3. does axis only works for x, y, xe, ye?
4. would you mind listing the visual channels that support legend?

[sehilyi]

1. point, rect, bar, triangle-*.
2. That is correct. (I think we can decide now whether to move some of the channels to the style option). What do you think of moving opacity, background, and even strokeWidth to the style option?
3. axis is used in x and y
4. Currently, only row and color whey they are encoded with nominal values. Will need to add more legend.

[wangqianwen0418]

Thanks for the reply!
I think we can safely move background and strokeWidth to the style option. Note sure about opacity, I feel sometimes I may want to encode the opacity of points.
But we can move opacity to style for the current version. No need to support all functions at the first version

[sehilyi]

Good suggestion. In that case, I think we can move only background to the style, and support encoding quantitative values to opacity and strokeWidth, which only requires me to add a few lines. I think it does not hurt to have that. I've added an issue for this (#39).

[wangqianwen0418]

Another question:
is compositeMark still supported? or you plan to replace it with superpose?

[sehilyi]

Surprised that you found this :)

Although I called it compositeMark, it is more like a reusable template for custom tracks. It is not being used in the grammar currently (once supported but removed temporally to focus on implementing other features), but I am thinking of adding an option in Track (like template) and moving the functionality of compositeMark to this option.

[wangqianwen0418]

Hi @sehilyi, I am trying to make a table (like the table below) in the wiki docs.

mark type	supported visual channels
point	x, y, size, color, strokeWidth, opacity
line	x, y, color, strokeWidth, opacity
rect	x, xe, color, strokeWidth, opacity
bar	x, y, ...
area	x, y, ...
...	...

Since you are more familiar with the code base, would you mind helping me fulfill the skeleton of the table (if you think this table is useful)?

[wangqianwen0418]

Also, I am planning to make two tables about the visual channel properties:

---

properties shared by all visual channels

visual channel properties	type	description
field	string	the data field name
type	string	specify type of the data field. support "genomic", "nominal", "quantitative"
aggregate	string	support "max", "min", "mean", "bin", "count"
domain
range

properties that are only used by certain visual channels

visual channel properties	type	supported visual channels	description
axis	string	x, y	It specifies the position of the axis. Support "none", "top", "bottom", "left", "right"
baseline
legend

[sehilyi]

They look great! I will update the <supported visual channels> table directly in the wiki.

[wangqianwen0418]

Many thanks for updating the supported visual channel table~

Just one question, it seems size is also supported in the line mark (and some other marks that don't include size in the table, I haven't check all of them). here

[wangqianwen0418]

I am not very sure about the rule mark, could you give more explanation?

[sehilyi]

Currently, it is only used for showing segmented lines (w/ diverse styles) that represent genomic intervals (like in gene annotation tracks). But, I am considering to either (1) merge its encoding functionalities to rect since they are similar each other or (2) add more unique encodings to rule. (Originally, it was for showing vertical or horizontal lines.)

Since work is left for this mark type, how about leaving this part empty in the docs for now and revisit this later after implementation.

[wangqianwen0418]

Great, I agree. I will leave this part empty.

[wangqianwen0418]

In the current grammar, users define the layout of a track using "circularLayout": true/false. Maybe "trackLayout": "circular"/"linear" makes more sense?

[sehilyi]

I agree, and I was thinking of renaming some properties for the consistency, including circularLayout and static. I've just added a related issue (#59).

[wangqianwen0418]

One more thing, what do you think about adding a render button in the online editor.
The right side only updates after the button is clicked

The current online editor renders whenever there is a change in the left side. Sometimes, the whole webpage crashes due to some rendering error while I am modifying the configuration. That is a little bit annoying.

[sehilyi]

Hi Qianwen, thanks for the suggestion. I was thinking about adding a mode-change button ("manual" <=> "auto") for the use case.

Also, can you share me what kinds of changes on the code resulted in the app crash? It looks like a bug, it should show the following message rather than the app crash:

FYI, I've fixed a bug that resulted in the app crash three days ago (#52). If you confronted the issue since then, please let me know.

[wangqianwen0418]

Oh, it is exactly the bug you fixed at #52~ Thanks!

[sehilyi]

Hi @wangqianwen0418, FYI, you can now generate unique urls (#51) to display custom specs in the editor. This can be useful when providing links for individual examples (i.e., "Open the example in online editor" in the current docs). To obtain the url, edit the spec as you want and click on the button on the top-right corner in the editor. Then, a unique url will be copied to your clipboard.

The example url looks like this:

Open the example in online editor

[wangqianwen0418]

Some question about bar:
how the width of a bar is calculated?
can user set the gap between bars?
does x indicates the left side of a bar or the centre of a bar?

many thanks~

[sehilyi]

x indicates the center position of bars.

• You could use both x and xe to bind start and end position of bars to a certain genomic interval.
• You could also use size to make the width of bars being constant values (e.g., size: { value: 6} == width of bars is 6px).

[wangqianwen0418]

This seems to be a small bug. The annotation of the y axis is wired.

Should I start an issue?

[sehilyi]

Yes, it would be really helpful if you write issues whenever you find unexpected behaviors

[sehilyi]

I guess that axis appeared when you set axis: "left"? It would be great if you could put a spec that made this behavior

[wangqianwen0418]

I am a little confused about triangle-d. Is it actually triangle-b, i.e., triangle bottom?

[sehilyi]

It refers to "down", but b for bottom might be a more consistent term

[sehilyi]

@wangqianwen0418 Do you think it is better to use "triangle-up", "triangle-down", "triangle-right", or "triangle-left" as in Vega? (Rather than the short names, like "triangle-l").

[wangqianwen0418]

@sehilyi , when you are convenient, would you mind taking a look at the current wiki documentations? You comments are welcome!

I haven't start writing the section data and the table properties that are only used by certain visual channels , so you can just ignore those parts.

It would be great if you can point me to some examples of the semantic zoom and tooltip.

Do you think we have some examples of using triangle marks without introducing superpose? I try to introduce the types of mark before superpose but find it is hard to achieve

[sehilyi]

Hi @wangqianwen0418, thank you for support, I will look into the documentation.

I think the current best example for triangle w/o using superpose would be showing gene position, like this one. Even better one would be showing a 90-deg rotated heatmap (#33), which is not yet supported in the grammar.

Regarding semantic zoom, the upper three track definitions in the "Semantic Zooming" demo will be good such examples. If you have question regarding the examples, please let me know.

The tooltip example is here (Please find the tooltip: {...} definition in the spec. And please note that, currently, the functionality of tooltip is quite limited as only rect marks in linear layouts are supported for showing tooltips.)

[wangqianwen0418]

Many thanks!

[wangqianwen0418]

Since the current support for tooltip is limited, do you think we should remove it from the current documentation?

[sehilyi]

@wangqianwen0418 I've briefly looked into the documentation and made some changes, but it looks good! I can see that people can already understand many parts of the grammar by reading the docs.

Thinks we may need to consider adding in the future would be (1) describing visual parameters for circular layouts (e.g., startAngle, endAngle, innerRadius, outerRadius, ...) and (2) little more details about field/channel (e.g., field.type, channel.domain, channel.range).

Also, as documentation covers many parts already, it would be worth discussing about static sites at some point.

[wangqianwen0418]

Do you think the triggerCondition interface can be refactored in the following way:

interface TriggerCondition {
  operation: LogicalOperation,
  measure: "width"|"height"|"zoomLevel",
  threshold: number|"|xe-x|",
  target: 'track' | 'mark' | 'superpose',
  conditionPadding?:number,
  transitionPadding?:number
}

this interface looks more clear to me.
But I totally understand that you might have other design considerations

[wangqianwen0418]

Also, I can create a pull request for this modification (later this week) if it makes sense to you

[sehilyi]

I like the renaming "condition" => "measure" & "threshold" which I think is more understandable. I also like the idea to force users to select a single measure.

One thing we could additionally do would be separating the interface based on the measure since its values (width, height, zoomLevel) use different properties. So something like:

...
visibility?: VisibilityCondition;
...

type VisibilityCondition = SizeVisibilityConsition | ZoomLevelVisibilityCondition;

interface CommonVisibilityCondition {
  operation: LogicalOperation;
  target: 'track' | 'mark'
}

interface SizeVisibilityCondition extends CommonVisibilityCondition {
  measure: 'width' | 'height';
  threshold: number | '|xe-x|;
  conditionPadding?: number;
  transitionPadding?: number;
}

interface ZoomLevelVisibilityCondition extends CommonVisibilityCondition {
  measure: 'zoomLevel';
  threshold: number;
}

We could add an issue for this and update the code in a PR. It would be helpful if you can help on this.

[wangqianwen0418]

Cool, I have created an issue #124~

[wangqianwen0418]

Just to clarify, the current dataTransform only supports three types of filters: oneOf, inRange, include, right?

Another question, do you think it makes sense to assign not a default false value in the filter?

[sehilyi]

That is correct, and I think making the not an optional value (not?: boolean) and considering it as false by default if it is undefined looks like a good idea.

[wangqianwen0418]

Cool~ I created a pull request.
I have also updated the wiki documents about semantic zoom, dataTransform, and add some details about the circular layout

[sehilyi]

Just read the documentation, and those parts are great!

[wangqianwen0418]

Just to clarify, does this mean that csv data can only be loaded from url while json data can only be loaded from inplace json value?
If a user has a csv file on his/her computer, how to access the csv file in Gosling?

export interface CSVDataGosling extends DataDeepGoslingCommon {
    type: 'csv';
    url?: string;
    separator?: string;
    // experimental
    headerNames?: string[];
    chromosomePrefix?: string;
    longToWideId?: string;
}

export interface JSONDataGosling extends DataDeepGoslingCommon {
    type: 'json';
    values?: Datum[];
}

Also, could you point me to some examples using JSON data?
As Nils mentioned, I think it is important that we have data that is more accessible and easy to read, so that users can know how to play with their own data.

[sehilyi]

Just to clarify, does this mean that csv data can only be loaded from url while json data can only be loaded from inplace json value?

That is correct.

[sehilyi]

If a user has a csv file on his/her computer, how to access the csv file in Gosling?

As a JSON-format grammar itself, we cannot support that. Since our grammar uses a JSON format as an input, raw data can be either (1) specified in a JSON format as a part of the spec or (2) stored at somewhere while its url is provided in the spec (I think this is the same in Vega/Vega-Lite).

When we build a wrapper library in the future (e.g., Gosling-React or Gosling.py), we could support uploading a local csv file. However, I think our main focus would be on the grammar itself until the paper submission.

Perhaps, an easiest way to allow people to play with their csv files would be supporting it in the editor (so outside of the grammar), e.g., a user upload a csv file, the editor converts it into a json format, and put it into a spec using a json type data spec.

One thing worth mentioning here is that our focus is more on the genomic-specific file formats than those generic formats, and, because such genomic data is commonly large, allowing people to upload the local file directly to a system is quite challenging. (HiGlass currently only support specifying urls of files for this reason, and no local files supported). We will need to think about more on how to allow people to load their local data for the more accessible and easier grammar usage, which I think is a really important issue.

As Nils mentioned, I think it is important that we have data that is more accessible and easy to read, so that users can know how to play with their own data.

Yes, I totally agree. I think we can prepare datasets (both in AWS buckets and Gosling's public HiGlass servers) and provide a list of datasets with public urls and detailed information so that people can try our grammar with them more accessibly. We can also add tutorials using small csv/json files to more easily give a sense on how the data can be used in the grammar for visual encoding.

[sehilyi]

@wangqianwen0418 I have a question that is perhaps related to this thread:

Do you think it will be largely helpful for users to have a Data Table panel in the editor where people can check how the raw data looks? I think this could help on understanding what is the role of metadata option as well.

[wangqianwen0418]

Hi Sehi, so many thanks for these information!

I think we will be using metadata at least until our version 1 release. This is a required option especially for converting complex genomic data formats into tabular formats.

The timeline looks good.

For the JSON & CSV data format, I would not worry too much about showing them in the editor (they are not bad in terms of readability). Users can always download the csv file from the url and open it on their own computer. But I do agree that examples using csv data are important for the tutorial.
If you do not have enough time, maybe we can prioritise csv examples over json.
Inserting json data makes the code very long. And json doesn't much more readability than csv file.

For table panel & file upload features, yes, they are quite important.
On the other hand, I think examples with csv files would be great enough for the twitter release.
CSV examples enable people to understand the data and play with the example.
Meanwhile, metaData examples show that Gosling has the ability to process more complicated genomic-specific file

[sehilyi]

On the other hand, I think examples with csv files would be great enough for the twitter release.

We can discuss further tomorrow, but I think it would be still important to prepare enough number/diversity of examples with genomic-specific files since we want to give impression that the grammar can support complex and large genomic data.