Skip to content

Commit d26ebf9

Browse files
deploy: 1e755c6
0 parents  commit d26ebf9

173 files changed

Lines changed: 79078 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.nojekyll

Whitespace-only changes.

CNAME

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
nbdev.fast.ai

api/clean.html

Lines changed: 1057 additions & 0 deletions
Large diffs are not rendered by default.

api/clean.html.md

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
# clean
2+
3+
4+
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
5+
6+
To avoid pointless conflicts while working with jupyter notebooks (with
7+
different execution counts or cell metadata), it is recommended to clean
8+
the notebooks before committing anything (done automatically if you
9+
install the git hooks with `nbdev-install-hooks`). The following
10+
functions are used to do that. Cleaning also adds cell `id`s if missing
11+
(required by nbformat 4.5+).
12+
13+
## Trust
14+
15+
------------------------------------------------------------------------
16+
17+
<a
18+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L25"
19+
target="_blank" style="float:right; font-size:smaller">source</a>
20+
21+
### nbdev_trust
22+
23+
``` python
24+
25+
def nbdev_trust(
26+
fname:str=None, # A notebook name or glob to trust
27+
force_all:bool=False, # Also trust notebooks that haven't changed
28+
):
29+
30+
```
31+
32+
*Trust notebooks matching `fname`.*
33+
34+
## Clean
35+
36+
------------------------------------------------------------------------
37+
38+
<a
39+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L87"
40+
target="_blank" style="float:right; font-size:smaller">source</a>
41+
42+
### clean_nb
43+
44+
``` python
45+
46+
def clean_nb(
47+
nb, # The notebook to clean
48+
clear_all:bool=False, # Remove all cell metadata and cell outputs?
49+
allowed_metadata_keys:list=None, # Preserve the list of keys in the main notebook metadata
50+
allowed_cell_metadata_keys:list=None, # Preserve the list of keys in cell level metadata
51+
clean_ids:bool=True, # Remove ids from plaintext reprs?
52+
):
53+
54+
```
55+
56+
*Clean `nb` from superfluous metadata*
57+
58+
Jupyter adds a trailing <code></code> to images in cell outputs.
59+
Vscode-jupyter does not.
60+
Notebooks should be brought to a common style to avoid unnecessary
61+
diffs:
62+
63+
``` python
64+
test_nb = read_nb('../../tests/image.ipynb')
65+
assert test_nb.cells[0].outputs[0].data['image/png'][-1] == "\n" # Make sure it was not converted by acccident
66+
clean_nb(test_nb)
67+
assert test_nb.cells[0].outputs[0].data['image/png'][-1] != "\n"
68+
```
69+
70+
The test notebook has metadata in both the main metadata section and
71+
contains cell level metadata in the second cell:
72+
73+
``` python
74+
test_nb = read_nb('../../tests/metadata.ipynb')
75+
76+
assert {'meta', 'jekyll', 'my_extra_key', 'my_removed_key'} <= test_nb.metadata.keys()
77+
assert {'meta', 'hide_input', 'my_extra_cell_key', 'my_removed_cell_key'} == test_nb.cells[1].metadata.keys()
78+
```
79+
80+
After cleaning the notebook, all extra metadata is removed, only some
81+
keys are allowed by default:
82+
83+
``` python
84+
clean_nb(test_nb)
85+
86+
assert {'jekyll', 'kernelspec'} == test_nb.metadata.keys()
87+
assert {'hide_input'} == test_nb.cells[1].metadata.keys()
88+
```
89+
90+
We can preserve some additional keys at the notebook or cell levels:
91+
92+
``` python
93+
test_nb = read_nb('../../tests/metadata.ipynb')
94+
clean_nb(test_nb, allowed_metadata_keys={'my_extra_key'}, allowed_cell_metadata_keys={'my_extra_cell_key'})
95+
96+
assert {'jekyll', 'kernelspec', 'my_extra_key'} == test_nb.metadata.keys()
97+
assert {'hide_input', 'my_extra_cell_key'} == test_nb.cells[1].metadata.keys()
98+
```
99+
100+
Passing `clear_all=True` removes everything from the cell metadata:
101+
102+
``` python
103+
test_nb = read_nb('../../tests/metadata.ipynb')
104+
clean_nb(test_nb, clear_all=True)
105+
106+
assert {'jekyll', 'kernelspec'} == test_nb.metadata.keys()
107+
test_eq(test_nb.cells[1].metadata, {})
108+
```
109+
110+
Passing `clean_ids=True` removes `id`s from plaintext repr outputs, to
111+
avoid notebooks whose contents change on each run since they often lead
112+
to git merge conflicts. For example:
113+
114+
<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28 at 0x7FB4F8979690>
115+
116+
becomes:
117+
118+
<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28>
119+
120+
*Cell* IDs, on the other hand, are always added if missing
121+
122+
``` python
123+
test_cell = {'source': 'x=1', 'cell_type': 'code', 'metadata': {}}
124+
_clean_cell(test_cell, False, set(), True)
125+
test_cell['id']
126+
```
127+
128+
'b3c0571e'
129+
130+
------------------------------------------------------------------------
131+
132+
<a
133+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L112"
134+
target="_blank" style="float:right; font-size:smaller">source</a>
135+
136+
### process_write
137+
138+
``` python
139+
140+
def process_write(
141+
warn_msg, proc_nb, f_in, f_out:NoneType=None, disp:bool=False
142+
):
143+
144+
```
145+
146+
*Call self as a function.*
147+
148+
------------------------------------------------------------------------
149+
150+
<a
151+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L135"
152+
target="_blank" style="float:right; font-size:smaller">source</a>
153+
154+
### nbdev_clean
155+
156+
``` python
157+
158+
def nbdev_clean(
159+
fname:str=None, # A notebook name or glob to clean
160+
clear_all:bool=False, # Remove all cell metadata and cell outputs?
161+
disp:bool=False, # Print the cleaned outputs
162+
stdin:bool=False, # Read notebook from input stream
163+
):
164+
165+
```
166+
167+
*Clean all notebooks in `fname` to avoid merge conflicts*
168+
169+
By default (`fname` left to `None`), all the notebooks in
170+
`config.nbs_path` are cleaned. You can opt in to fully clean the
171+
notebook by removing every bit of metadata and the cell outputs by
172+
passing `clear_all=True`.
173+
174+
If you want to keep some keys in the main notebook metadata you can set
175+
`allowed_metadata_keys` in `[tool.nbdev]` in `pyproject.toml`. Similarly
176+
for cell level metadata use: `allowed_cell_metadata_keys`. For example,
177+
to preserve both `k1` and `k2` at both the notebook and cell level add
178+
the following to `pyproject.toml`:
179+
180+
``` toml
181+
[tool.nbdev]
182+
allowed_metadata_keys = ["k1", "k2"]
183+
allowed_cell_metadata_keys = ["k1", "k2"]
184+
```
185+
186+
------------------------------------------------------------------------
187+
188+
<a
189+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L150"
190+
target="_blank" style="float:right; font-size:smaller">source</a>
191+
192+
### clean_jupyter
193+
194+
``` python
195+
196+
def clean_jupyter(
197+
path, model, kwargs:VAR_KEYWORD
198+
):
199+
200+
```
201+
202+
*Clean Jupyter `model` pre save to `path`*
203+
204+
This cleans notebooks on-save to avoid unnecessary merge conflicts. The
205+
easiest way to install it for both Jupyter Notebook and Lab is by
206+
running `nbdev-install-hooks`. It works by implementing a
207+
`pre_save_hook` from Jupyter’s [file save hook
208+
API](https://jupyter-server.readthedocs.io/en/latest/developers/savehooks.html).
209+
210+
## Hooks
211+
212+
------------------------------------------------------------------------
213+
214+
<a
215+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L191"
216+
target="_blank" style="float:right; font-size:smaller">source</a>
217+
218+
### nbdev_install_hooks
219+
220+
``` python
221+
222+
def nbdev_install_hooks(
223+
224+
):
225+
226+
```
227+
228+
*Install Jupyter and git hooks to automatically clean, trust, and fix
229+
merge conflicts in notebooks*
230+
231+
See
232+
[`clean_jupyter`](https://nbdev.fast.ai/api/clean.html#clean_jupyter)
233+
and `nbdev-merge` for more about how each hook works.

0 commit comments

Comments
 (0)