Scimax has used a forked version ob-ipython for a long time. The upstream version has improved a lot over the last year, and is nicer than my forked version, especially for asynchronous blocks. So…. I have been working to integrate the upstream version and replace my forked version. Well, mostly. I have specific things I want in org-mode/ipython integration that aren’t built in to ob-ipython, and aren’t customizable either. So, my new integration still monkey patches quite a few ob-ipython functions.
The new update is in two files: ./scimax-ob.el, which contains pretty general functions for all org babel src blocks I think, and ./scimax-org-babel-ipython-upstream.el which contains monkey-patches on ob-ipython, and additional features.
This document shows the new features.
I am anticipating deprecating the old ob-ipython fork, but I have no idea how many people use it, and have no issue with it, so I have added an ob-ipython-upstream submodule, and started a new scimax-org-babel-ipython-upstream.el library that is independent of the old one.
Source blocks usually differentiate between results that are output and value. This has rarely made sense for how I use org-mode. I usually print things and want outputs. Anything you print will show in the results, and the last value will also show. Printed output comes first.
def f(x):
print(x)
return x**2
f(1.41)
1.9880999999999998
A new feature for scimax users is the execution count, which shows you the counter for when the cell was executed. This shows as comment in the results, so it won’t appear in exported content. The count is controlled by the output of the function defined in ob-ipython-execution-count. See the docstring of that variable for other options.
(setq ob-ipython-execution-count 'ob-ipython-execution-count-suppress)Ipython can have many kinds of output. The # text/plain line in the output tells you what kind of output this is. Later we will see how to filter these. If you don’t like these lines, turn them off like this.
(setq ob-ipython-show-mime-types nil)The output really shines with printed output and figures. In the upstream ob-ipython you cannot do both, and it only shows one figure. We get everything with scimax, including all the outputs. You can also filter the outputs to only show what you want.
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
t = np.linspace(0, 20 * np.pi, 350)
x = np.exp(-0.1 * t) * np.sin(t)
y = np.exp(-0.1 * t) * np.cos(t)
plt.plot(x, y)
plt.axis('equal')
plt.figure()
plt.plot(y, x)
plt.axis('equal')
print('Length of t = {}'.format(len(t)))
print('x .dot. y = {}'.format(x @ y))
You can use an :ipyfile header argument to put attributes on images in the output and to specify their filenames. This is helpful when writing technical documents so you can do things like make references to them like ref:clockwise and ref:counterclockwise, and when you want nice filenames for other purposes. We support :name, :filename, :caption, :attr_org, :attr_html and :attr_latex. These should be entered as a list of plists. It is necessary to quote the list.
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
t = np.linspace(0, 20 * np.pi, 350)
x = np.exp(-0.1 * t) * np.sin(t)
y = np.exp(-0.1 * t) * np.cos(t)
plt.plot(x, y)
plt.axis('equal')
plt.figure()
plt.plot(y, x)
plt.axis('equal')
print('Length of t = {}'.format(len(t)))
print('x .dot. y = {}'.format(x @ y))
By default we get all the outputs from the kernel. Here we get plain text, an image and a LaTeX representation.
from sympy import *
init_printing()
x, y, z = symbols('x y z')
Integral(sqrt(1 / x), x)
You can choose to display in the src block header by specifying which mime-type to display (that is one reason I added it to the output).
Integral(sqrt(1/x), x)
I have not added an exclude feature, but it would not be hard to. Unfortunately, it is not yet possible to control the order these come out in, or to do things like add captions or size information to the images.
By default we capture exceptions in results. I like that because for notes, it is nice to show what happens, and I find it less disruptive while working to not have windows opening and closing.
print(1 / 0)
ZeroDivisionErrorTraceback (most recent call last) <ipython-input-7-3ec96714f820> in <module>() —-> 1 print(1 / 0)
ZeroDivisionError: division by zero
If you like an exception buffer, set this variable like this:
(setq ob-ipython-exception-results nil)Even this is better, press q in that buffer to jump to the offending line in your src block. This does not work right when you run in async mode.
print(1 / 0)
You can now access the documentation like you can in jupyter.
import numpy as np
?np.linspace
Similarly you can see the source code:
??np.linspace
This works for functions defined in your org file too.
def func(X):
'''a function of X'''
return 2 * X
?func
You can inspect things in your src-blocks to find out things about them. With your cursor on some object, type s-/ and if it can be figured out you will get some information about it.
import numpy as np
x = np.linspace(0, 1, 5)
x
You can also enable something like eldoc with elisp:scimax-ob-ipython-turn-on-eldoc and turn it off with elisp:scimax-ob-ipython-turn-off-eldoc. This is a little tricky, it sometimes only works when you have a syntactically correct command with parentheses, or after some cursor movement. I wish it worked better.
np.linspace
You can get some completion options. I like scimax-ob-ipython-complete-ivy. I have this bound to s-. It often works on objects too.
np.linalg.add
x.argsort
You can also use company mode like this:
(add-to-list 'company-backends 'company-ob-ipython)
(company-mode)np.linalg.inv
Company-mode is kind of slow, and lacks the completion like ivy, but it might work for you.
You can use an :async header to run a block asynchronously. That means it runs in the background and you can keep using emacs! You can mix and match async blocks in a document. I simplified how this is done compared to upstream; in my version just putting :async in the header (with no argument) makes it run asynchronously.
import time
for i in range(4):
print(i)
time.sleep(2)
# type new things
# keep on working!
print('done')
You will see another buffer pop up with intermediate results, and they will be put back in the results when it is done.
ipython/org-mode really shines when you start leveraging rich outputs from Ipython. A new feature I have added is that you can write your own functions to customize the output.
This variable maps mime-types to formatting functions. You can add new mime-types to this, or redefine the formatting functions if you don’t like the way the work.
(append '(("mime-type" "formatting function"))
'(hline)
(loop for (mime-type . func) in ob-ipython-mime-formatters
collect (list mime-type func)))| mime-type | formatting function |
|---|---|
| text/plain | ob-ipython-format-text/plain |
| text/html | ob-ipython-format-text/html |
| text/latex | ob-ipython-format-text/latex |
| text/org | ob-ipython-format-text/org |
| image/png | ob-ipython-format-image/png |
| image/svg+xml | ob-ipython-format-image/svg+xml |
| application/javascript | ob-ipython-format-application/javascript |
| default | ob-ipython-format-default |
| output | ob-ipython-format-output |
You can set these to whatever you want, and add new ones for new mimetypes.
Most python objects have a __str__ or __repr__ method defined that display them when printed. For example, here is a Polynomial from numpy with it’s default representation.
import numpy as np
p = np.polynomial.Polynomial([1, 2, 3])
p
Let’s change this to get a LaTeX representation (adapted from https://github.com/jupyter/ngcm-tutorial/blob/master/Part-1/IPython%20Kernel/Custom%20Display%20Logic.ipynb). We will do this on the Ipython side of output customization where we register a formatting function for a specific type in IPython.
def poly_to_latex(p):
terms = ['%.2g' % p.coef[0]]
if len(p) > 1:
term = 'x'
c = p.coef[1]
if c != 1:
term = ('%.2g ' % c) + term
terms.append(term)
if len(p) > 2:
for i in range(2, len(p)):
term = 'x^%d' % i
c = p.coef[i]
if c != 1:
term = ('%.2g ' % c) + term
terms.append(term)
px = '$P(x)=%s$' % '+'.join(terms)
dom = r', $x \in [%.2g,\ %.2g]$' % tuple(p.domain)
return px + dom
ip = get_ipython()
latex_f = ip.display_formatter.formatters['text/latex']
latex_f.for_type_by_name('numpy.polynomial.polynomial',
'Polynomial', poly_to_latex)
p
That looks nice, but we can go one step further and define graphical outputs too.
import matplotlib.pyplot as plt
from IPython.core.pylabtools import print_figure
def poly_to_png(p):
fig, ax = plt.subplots()
x = np.linspace(-1, 1)
y = [p(_x) for _x in x]
ax.plot(x, y)
ax.set_title(poly_to_latex(p))
ax.set_xlabel('x')
ax.set_ylabel('P(x)')
data = print_figure(fig, 'png')
# We MUST close the figure, otherwise IPython's display machinery
# will pick it up and send it as output, resulting in a double display
plt.close(fig)
return data
ip = get_ipython()
png_f = ip.display_formatter.formatters['image/png']
png_f.for_type_by_name('numpy.polynomial.polynomial',
'Polynomial', poly_to_png)
Now, we can easily see the formula and shape of this polynomial in a graphical form.
p
Most likely you would not put all this code into a document like this, but would instead put it in a Python library you import. The point here is to show what can be done with that, and once it is done, you get easy visualization of objects.
In Tensorflow, we are always making computation graphs. These are usually visualized in Tensorboard. We can leverage Jupyter to show us a graphical representation instead. This is another example of registering a type in Ipython.
from graphviz import Digraph
def tf_to_dot(graph):
"Adapted from https://blog.jakuba.net/2017/05/30/tensorflow-visualization.html"
dot = Digraph()
for n in g.as_graph_def().node:
dot.node(n.name, label=n.name)
for i in n.input:
dot.edge(i, n.name)
dot.format = 'svg'
return dot.pipe().decode('utf-8')
ip = get_ipython()
svg_f = ip.display_formatter.formatters['image/svg+xml']
svg_f.for_type_by_name('tensorflow.python.framework.ops',
'Graph', tf_to_dot)
import tensorflow as tf
g = tf.Graph()
with g.as_default():
a = tf.placeholder(tf.float32, name="a")
b = tf.placeholder(tf.float32, name="b")
c = a + b
g
Now we have a record of what the graph looks like. Ez peezy.
Just for fun, here is a way to get Pandas dataframes to be displayed as org-mode tables using tabulate (https://pypi.python.org/pypi/tabulate). This is adapted from https://github.com/gregsexton/ob-ipython. tabulate has a built-in org formatter, so no reason to reinvent that!
import IPython
import tabulate
class OrgFormatter(IPython.core.formatters.BaseFormatter):
format_type = IPython.core.formatters.Unicode('text/org')
print_method = IPython.core.formatters.ObjectName('_repr_org_')
def pd_dataframe_to_org(df):
return tabulate.tabulate(df, headers='keys', tablefmt='orgtbl', showindex='always')
ip = get_ipython()
ip.display_formatter.formatters['text/org'] = OrgFormatter()
f = ip.display_formatter.formatters['text/org']
f.for_type_by_name('pandas.core.frame', 'DataFrame', pd_dataframe_to_org)
import pandas as pd
df = pd.DataFrame([1, 2], columns=['widecolumn'])
df.index.name = 'indexname'
df
| indexname | widecolumn |
|---|---|
| 0 | 1 |
| 1 | 2 |
Here is a bigger example.
import numpy as np
df2 = pd.DataFrame(np.random.randint(low=0, high=10, size=(5, 5)),
columns=['a', 'b', 'c', 'd', 'e'])
df2
| a | b | c | d | e | |
|---|---|---|---|---|---|
| 0 | 6 | 2 | 4 | 3 | 9 |
| 1 | 6 | 1 | 3 | 9 | 8 |
| 2 | 7 | 9 | 7 | 0 | 0 |
| 3 | 6 | 2 | 2 | 3 | 9 |
| 4 | 3 | 1 | 9 | 9 | 8 |
The canonical way to make rich outputs on your own classes is to add repr_X methods. Here is the example from the Jupyter tutorial listed above. Here, we just add a PNG and LaTeX representation.
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
from IPython.core.pylabtools import print_figure
from IPython.display import Image, SVG, Math
class Gaussian(object):
"""A simple object holding data sampled from a Gaussian distribution.
"""
def __init__(self, mean=0.0, std=1, size=1000):
self.data = np.random.normal(mean, std, size)
self.mean = mean
self.std = std
self.size = size
# For caching plots that may be expensive to compute
self._png_data = None
def _figure_data(self, format):
fig, ax = plt.subplots()
ax.hist(self.data, bins=50)
ax.set_title(self._repr_latex_())
ax.set_xlim(-10.0,10.0)
data = print_figure(fig, format)
# We MUST close the figure, otherwise IPython's display machinery
# will pick it up and send it as output, resulting in a double display
plt.close(fig)
return data
def _repr_png_(self):
if self._png_data is None:
self._png_data = self._figure_data('png')
return self._png_data
def _repr_latex_(self):
return r'$\mathcal{N}(\mu=%.2g, \sigma=%.2g),\ N=%d$' % (self.mean,
self.std, self.size)
Gaussian()
We can define custom outputs for our own objects too. Here we define org and html representations of a heading object within the class. We have to define a repr_mimebundle method to get ‘text/org’ output as it is not a predefined type in Jupyter. Alternatively, we could use the methods earlier to define formatters for these types.
See http://nbviewer.jupyter.org/github/ipython/ipython/blob/6.x/examples/IPython%20Kernel/Custom%20Display%20Logic.ipynb#Custom-Mimetypes-with-_repr_mimebundle_ for more details.
class Heading(object):
def __init__(self, content, level=1, tags=()):
self.content = content
self.level = level
self.tags = tags
def _repr_org(self):
s = '*' * self.level + ' ' + self.content
if self.tags:
s += f" :{':'.join(self.tags)}:"
return s
def _repr_html(self):
return f"<h{self.level}>{self.content}</h{self.level}>"
def _repr_mimebundle_(self, include, exclude, **kwargs):
"""
repr_mimebundle should accept include, exclude and **kwargs
"""
data = {'text/html': self._repr_html(),
'text/org': self._repr_org()
}
if include:
data = {k:v for (k,v) in data.items() if k in include}
if exclude:
data = {k:v for (k,v) in data.items() if k not in exclude}
return data
Now, you can construct headings in iPython, and get different outputs that might be suitable for different purposes.
Heading('A level 4 headline', level=4, tags=['example'])
The Jupyter notebook does really shine for JavaScript driven interactive data exploration. For now, the only option for Emacs is to open external programs for this, e.g. a matplotlib figure, or a browser. Bokeh is a really interesting interactive plotting library you can use in Python, but it makes interactive html documents for viewing in a browser. Here we will adapt the outputs to show us a thumbnail and org-link to open the html file. This is yet another example of registering a type in Ipython.
Here we modify the plain text output so that it saves an html file, and returns a link to it.
Note you need to install bokeh, selenium, pillow with conda, and install a modern phantomjs in your OS for this to work (I build one from https://github.com/eisnerd/phantomjs).
import tempfile
import warnings
warnings.filterwarnings("ignore")
import webbrowser
import IPython
from bokeh.io import export_png
from bokeh.io.saving import save
from bokeh.plotting.figure import Figure
import os
def bokeh_to_org(plt):
fh, tmp = tempfile.mkstemp(suffix=".html", prefix="bokeh-",
dir="obipy-resources")
fname = save(plt, tmp)
os.close(fh)
os.system(f'open {fname}')
return "[[{}]]".format(fname)
def bokeh_to_png(plt):
png_filename = export_png(plt)
with open(png_filename, "rb") as f:
return f.read()
def bokeh_mimebundle(self, include=(), exclude=(), **kwargs):
data = {'text/org': bokeh_to_org(self),
'image/png': bokeh_to_png(self)}
if include:
data = {k:v for (k,v) in data.items() if k in include}
if exclude:
data = {k:v for (k,v) in data.items() if k not in exclude}
return data, {}
Figure._repr_mimebundle_ = bokeh_mimebundle
Now we are setup to make an interactive figure.
from bokeh.io import output_file, show
from bokeh.models import ColumnDataSource, HoverTool
from bokeh.sampledata.periodic_table import elements
from bokeh.transform import dodge, factor_cmap
periods = ["I", "II", "III", "IV", "V", "VI", "VII"]
groups = [str(x) for x in range(1, 19)]
df = elements.copy()
df["atomic mass"] = df["atomic mass"].astype(str)
df["group"] = df["group"].astype(str)
df["period"] = [periods[x - 1] for x in df.period]
df = df[df.group != "-"]
df = df[df.symbol != "Lr"]
df = df[df.symbol != "Lu"]
cmap = {
"alkali metal": "#a6cee3",
"alkaline earth metal": "#1f78b4",
"metal": "#d93b43",
"halogen": "#999d9a",
"metalloid": "#e08d49",
"noble gas": "#eaeaea",
"nonmetal": "#f1d4Af",
"transition metal": "#599d7A",
}
source = ColumnDataSource(df)
p = Figure(
title="Periodic Table (omitting LA and AC Series)",
plot_width=1000,
plot_height=450,
tools="",
toolbar_location=None,
x_range=groups,
y_range=list(reversed(periods)))
p.rect(
"group",
"period",
0.95,
0.95,
source=source,
fill_alpha=0.6,
legend="metal",
color=factor_cmap(
"metal", palette=list(cmap.values()), factors=list(cmap.keys())))
text_props = {"source": source, "text_align": "left", "text_baseline": "middle"}
x = dodge("group", -0.4, range=p.x_range)
r = p.text(x=x, y="period", text="symbol", **text_props)
r.glyph.text_font_style = "bold"
r = p.text(
x=x,
y=dodge("period", 0.3, range=p.y_range),
text="atomic number",
**text_props)
r.glyph.text_font_size = "8pt"
r = p.text(
x=x, y=dodge("period", -0.35, range=p.y_range), text="name", **text_props)
r.glyph.text_font_size = "5pt"
r = p.text(
x=x,
y=dodge("period", -0.2, range=p.y_range),
text="atomic mass",
**text_props)
r.glyph.text_font_size = "5pt"
p.text(
x=["3", "3"],
y=["VI", "VII"],
text=["LA", "AC"],
text_align="center",
text_baseline="middle")
p.add_tools(
HoverTool(tooltips=[
("Name", "@name"),
("Atomic number", "@{atomic number}"),
("Atomic mass", "@{atomic mass}"),
("Type", "@metal"),
("CPK color", "$color[hex, swatch]:CPK"),
("Electronic configuration", "@{electronic configuration}"),
]))
p.outline_line_color = None
p.grid.grid_line_color = None
p.axis.axis_line_color = None
p.axis.major_tick_line_color = None
p.axis.major_label_standoff = 0
p.legend.orientation = "horizontal"
p.legend.location = "top_center"
p
Now if you click on the link above, it will open an interactive html file in your browser. It is just a tempfile, so some work might be necessary to get it to a persistent place, like the images are.
This example is also from this url, and shows how to override the IPython output all together.
import numpy as np
import json
import uuid
from IPython.display import display_javascript, display_html, display
class FlotPlot(object):
def __init__(self, x, y):
self.x = x
self.y = y
self.uuid = str(uuid.uuid4())
def _ipython_display_(self):
json_data = json.dumps(list(zip(self.x, self.y)))
display_html('<div id="{}" style="height: 300px; width:80%;"></div>'.format(self.uuid),
raw=True)
display_javascript(f'''require(["//cdnjs.cloudflare.com/ajax/libs/flot/0.8.2/jquery.flot.min.js"], function() {{
var line = JSON.parse("{json_data}");
console.log(line);
$.plot("#{self.uuid}", [line]);
}});''', raw=True)
x = np.linspace(0,10)
y = np.sin(x)
FlotPlot(x, np.sin(x))
require(["//cdnjs.cloudflare.com/ajax/libs/flot/0.8.2/jquery.flot.min.js"], function() {
var line = JSON.parse("[[0.0, 0.0], [0.20408163265306123, 0.20266793654820095], [0.40816326530612246, 0.39692414892492234], [0.6122448979591837, 0.5747060412161791], [0.8163265306122449, 0.7286347834693503], [1.0204081632653061, 0.8523215697196184], [1.2244897959183674, 0.9406327851124867], [1.4285714285714286, 0.9899030763721239], [1.6326530612244898, 0.9980874821347183], [1.836734693877551, 0.9648463089837632], [2.0408163265306123, 0.8915592304110037], [2.2448979591836737, 0.7812680235262639], [2.4489795918367347, 0.6385503202266021], [2.6530612244897958, 0.469329612777201], [2.857142857142857, 0.28062939951435684], [3.0612244897959187, 0.0802816748428135], [3.2653061224489797, -0.12339813736217871], [3.4693877551020407, -0.3219563150726187], [3.673469387755102, -0.5071517094845144], [3.8775510204081636, -0.6712977935519321], [4.081632653061225, -0.8075816909683364], [4.285714285714286, -0.9103469443107828], [4.4897959183673475, -0.9753282860670456], [4.6938775510204085, -0.9998286683840896], [4.8979591836734695, -0.9828312039256306], [5.1020408163265305, -0.9250413717382029], [5.3061224489795915, -0.8288577363730427], [5.510204081632653, -0.6982723955653996], [5.714285714285714, -0.5387052883861563], [5.918367346938775, -0.35677924089893803], [6.122448979591837, -0.16004508604325057], [6.326530612244898, 0.04333173336868346], [6.530612244897959, 0.2449100710119793], [6.73469387755102, 0.4363234264718193], [6.938775510204081, 0.6096271964908323], [7.142857142857143, 0.7576284153927202], [7.346938775510204, 0.8741842988197335], [7.551020408163265, 0.9544571997387519], [7.755102040816327, 0.9951153947776636], [7.959183673469388, 0.9944713672636168], [8.16326530612245, 0.9525518475314604], [8.36734693877551, 0.8710967034823207], [8.571428571428571, 0.7534867274396376], [8.775510204081632, 0.6046033165061543], [8.979591836734695, 0.43062587038273736], [9.183673469387756, 0.23877531564403087], [9.387755102040817, 0.03701440148506237], [9.591836734693878, -0.1662827938487564], [9.795918367346939, -0.3626784288265488], [10.0, -0.5440211108893699]]");
console.log(line);
$.plot("#2ebc40f4-a6da-4ae4-b45f-78288681d551", [line]);
});Jupyter has some nice features like buttons to click on to run a block. We have something like that. The text in angle brackets in the comments below is clickable!
# <run> <restart and run> <repl> <delete block> <menu>
# open some buffers <output> <execute> <debug>
6 * 3
5
Jupyter notebooks have some nice key bindings, like all the variations of modified-return that do different things. When your cursor is in an ipython block, these bindings are active. They are not active outside of ipython code blocks. See this magical post for how that is possible!
| Ctrl-<return> | run current block |
| Shift-<return> | run current block and go to next one, create one if needed |
| Meta-<return> | runs the current cell and inserts a new one below. |
| super-<return> | restart ipython and run block |
| Meta-super-<return> | restart ipython and run all blocks to point |
| H-<return> | restart ipython and run all blocks in buffer |
5
1
2
3
Note you can put :restart in the src block header and ipython will restart every time you run that block. This is helpful when developing libraries, as it forces the library to be reloaded every time you run the block.
| H-= | insert src-block above current block |
| C-u hyper-= | insert src-block below current block |
| H– | split current block at point, point in upper block |
| C-u H– | split current block at point, point in lower block |
| H-h | Edit the src block header in the minibuffer |
| H-w | Kill the current block |
| H-n | Copy the current block |
| H-o | Clone the current block (make a copy of it below the current one |
| H-m | Merge blocks in the selected region |
| s-w | Move current block before the previous one |
| s-s | Move current block below the next one |
| H-l | Clear results from the block |
| H-s-l | Clear all results in buffer |
| s-i | Jump to previous block |
| s-k | Jump to next block |
| H-q | Jump to a visible block with avy |
| H-s-q | Jump to a block in the buffer with ivy |
| s-/ | get help about thing at point (ob-ipython-inspect) |
| H-r | switch to session REPL |
| H-k | Kill the kernel |
I am not 100% committed to all these bindings. I still find the need to fine tune them every once in a while.
Can’t remember all these bindings? Me neither. Checkout M-x scimax-obi/body (usually bound to H-s) for a nice hydra. The hydra key-bindings don’t match the ones in the tables above; I am not sure that makes sense. It would add keystrokes, but it would also be a good reminder of the bindings.
These keybindings are relatively easy to customize. The are stored as cons cells in this variable.
ob-ipython-key-bindingsYou can define/change any binding you want like this. They are only active in ipython src blocks. For example, you can define a src-block key like this:
(scimax-define-src-key ipython "H-/" #'some-command)If you like the Jupyter keybindings more directly, checkout these bindings when you are in an ipython block:
| H-e | function | scimax-jupyter-edit-mode/body | ||
| H-c | function | scimax-jupyter-command-mode/body |
Jupyter can run many languages ranging from Fortran to shell. I like hylang, a lispy Python. Install the hylang Jupyter kernel like this:
pip install git+https://github.com/Calysto/calysto_hy.git --user
python -m calysto_hy install --userNow we can use it in scimax. This is already pre-configured in scimax. For fun, we use hylang here.
(import [tensorflow :as tf])
(setv a (tf.constant 3)
b (tf.constant 3)
c (tf.add a b))
(with [sess (tf.Session)] (print (.run sess c)))
There is a little issue with the double colon on output, but otherwise it looks like it works.
This kernel is a little quieter on exceptions than I would like, but still it could be useful if you want to play around with hylang and document your work.
By default, each org file gets a unique kernel. I am sure there is a use case for every buffer sharing one kernel, but it is too confusing for me, and too prone to errors where one buffer changes a variable that affects others. So, if you want src blocks in different buffers to share kernels, you have to manually specify the kernel in the header, or use this for the original behavior.
(setq ob-ipython-buffer-unique-kernel nil)See this comment for an example of multiple remote sessions, and Remote kernels about remote kernels. Here we use a properties drawer in two different headlines to have different sessions open in the same org file. Of course, you can manually define the :session in src blocks, but using a header like this:
:PROPERTIES: :header-args:ipython: :session session-1 :END:
means that you don’t have to type that on every block. That is helpful, since if you forget the block will use the default buffer kernel. This shows I am currently running four kernels. See this comment for an example of multiple remote sessions.
(ob-ipython--get-kernel-processes)Every block under this header will use the kernel associated with session-1. Here we just look at what ports are being used.
%connect_info
Every block in this heading will use the kernel associated with session-2. You can see here it uses different ports than session-1 does. They are both running at the same time though.
%connect_info
According to a comment in issue 114 this finally works. I don’t have a remote server to test it on, but I did test it locally. Here are the steps to follow, adapted from https://vxlabs.com/2017/11/30/run-code-on-remote-ipython-kernels-with-emacs-and-orgmode/.
Note: This comment reports some flakiness with the method below, and suggests an alternative approach that they find more robust.
On the remote server, find out where your runtime directory is like this:
jupyter --runtime-dirStart a kernel on the remote server like this. It will create a json file that you will need to copy to your local machine jupyter runtime directory.
ipython kernelYou should see some text like this one.
NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.
To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.
To read more about this, see https://github.com/ipython/ipython/issues/2049
To connect another client to this kernel, use:
--existing kernel-16577.json
You can see the connection file is indeed in the runtime directory:
ls `jupyter --runtime-dir`On a real remote machine, you would have to copy this file to your local machine runtime directory. There is nothing too mysterious in this file, it just has some information about the IP address and ports used.
cat `jupyter --runtime-dir`/kernel-226309.jsonNow, on your local machine, connect to the remote kernel like this, obviously with your own username and hostname.
jupyter console --existing kernel-226309.json --ssh username@hostnameYou should get some output like:
[ZMQTerminalIPythonApp] Forwarding connections to 127.0.0.1 via [email protected] [ZMQTerminalIPythonApp] To connect another client via this tunnel, use: [ZMQTerminalIPythonApp] --existing kernel-16577-ssh.json Jupyter console 5.2.0 Python 3.6.3 |Anaconda, Inc.| (default, Oct 13 2017, 12:02:49) Type 'copyright', 'credits' or 'license' for more information IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.
The critical text to note here is --existing kernel-16577-ssh.json. You need to specify that kernel file (note the -ssh in it) as the :session in the header of the src block.
The command above opens a local Ipython terminal. In that terminal, I typed a = 7.
Then, you specify the kernel connection file as the :session argument in an ipython block.
print(a)
b = 6
You can see from the output that we successfully connected to the kernel and that indeed a=7. Furthermore, in the block above we assigned b=6, and in the terminal I can type “b” and see that it is equal to 6.
If you have a lot of blocks in your buffer, you can use a file property like this to specify the header for every block in the buffer. See Multiple sessions in one org file for an example of using heading properties instead.
#+PROPERTY: header-args:ipython :session kernel-16577-ssh.json
This seems to be an issue with the upstream repo. It may work on unix/Mac, but maybe not on Windows. Probably does not work on remote kernels. See https://github.com/gregsexton/ob-ipython/commit/fcb2c48f64ba1005ad6fe5d1a4149f1117a9b45d