Changelogs

0.7

DL:

• Added metab as extra metadata from dataloaders to Learner in the core training loop and modified Recorder to return it if available.

• Added n option to AutoIncrement.

• Added mask().

• Removed k1lib.stats(), replaced it with torch.Tensor.stats() directly

• Added op option to Fn.

• Added torch.Tensor.hasInfs()

• Made LossNLLCross integrates with AccF instead.

• Made AccF integrates with ConfusionMatrix.

• Made ConfusionMatrix resilient against nans and infs, and fixed norm strategy (tensor rows divide by max rows, instead of tensor columns divide by max rows, like before).

Cli:

• Added mask, unsqueeze().

• Made op compatible with lots of other cli tools, so you can do stuff like filt(op() > 3, 1).

• Removed split, as op().split().all() | joinStreams() does the same thing. This is longer, so I kinda want to put it back in, but the name is incredibly oxymoronic.

• Removed listToTable() and its alias tableFromList(), as they are too simple, and the new unsqueeze(1) does the same thing.

• Removed args option in applyMp, as it feels clunky.

• Removed cats as cat().all() does the same job.

• Sped up op to have roughly same performance as apply. As a consequence, removed [lstrip, rstrip, strip, upper, lower] clis as stuff like op().upper().all() does the same and I want to simplify the collection of cli tools.

• Completely removed k1lib.cli.ctx module, as the performance sacrifices are too much, and it doesn’t add a lot of useful features.

• Renamed utils.to1Str to join as that is more intuitive.

• Removed structural.table, as op().split().all() does the same thing

0.6

DL:

• Added LinBlock, Absorber.

• Added __module__ impersonation for wrapMod.

• Renamed LossLambda to LossF

• Renamed torch.nn.Module.preserveDevice() to torch.nn.Module.deviceContext() and add preserve buffers capability.

• Added torch.nn.Module.gradContext().

k1lib.selector.ModuleSelector:

• Renamed filter() to preprocess().

• Added hookFp(), hookF(), and hookB().

• Removed selected(), as __call__() does the same thing and is more intuitive

• Added prop option to named_children() and children().

• Removed callback Frozen as freeze() and unfreeze() does the job and is much more robust.

• Added freeze() and unfreeze()

• Removed copy(), as you can just grab another selector straight from the model real quick.

• Removed parameters(), as you can always access the associated torch.nn.Module inside and get params from there.

Cli:

• Added raiseError option in intercept.

• Added multi argument capabilities to __call__().

• Added op.

• Changed default param ignoreTensors of deref to True.

• Ignores Module by default in deref.

0.5

DL:

• Renamed k1lib.format to k1lib.fmt, and exposed it automatically.

• Renamed computation to comp, computationRate to compRate.

• Added sizeOf()

• Exposed selector automatically.

• Deleted k1lib.website

• Fixed infinity elements in landscape.Landscape’s plots.

• Split Accuracy’s accuracy calculating part into AccF.

Cli:

• Added chained mode to item

• Fixed ignoreTensor mechanic in deref

• Added prefetch mode in applyMp

• Added applyMpBatched that executes lots of transformations in a single job.

• Added clearPools() and pools().

• Added bytes reading and writing to cat file.

• Added activeSamples.

• Removed aliases splitColumns and joinColumns.

• Removed stitch, as to1Str().all() does the same thing.

0.4

This update moves things around a lot. The actual coding experience stays the same, but a lot of links < 0.4 will be broken in 0.4 docs. Go to 0.3 docs if you wish to follow links.

Molecule (k1lib.mo):

• Added close coulomb force calculations to simulator to make it more realistic (cyclohexane chair config possible now!)

• Added optional Hydrogen viewing to various functions in mo

• Fixed parsing bug where it doesn’t recognize “3-methyl” group in “3-methylpentane”.

DL:

• Don’t let AutoIncrement’s value property to auto increment internal value.

• Removed CyclicRandomSampler, as range(n) | repeatFrom() | randomize(n)` does the same thing.

• Removed FunctionDataset, added simpler FunctionData as replacement.

• Removed DataLoader, Data and DatasetWithSampler, as they’re complex and obsolete.

• Added tfImg(), tfFloat(), analyzeFloat()

• Monkey-patched torch.nn.Module to have piping capabilities.

• Renamed k1lib.data to k1lib.kdata, and exposed it automatically in imports.

• Renamed k1lib.nn to k1lib.knn, and exposed it automatically.

Bioinfo cli:

• Renamed k1lib.bioinfo.cli to k1lib.cli

• Added stagger() as the new interface for dataloaders.

• Added intercept, toPIL.

• Added PIL conversion mode to utils.toTensor

• Added Tensor shape mode in utils.shape.

• Added others module, with crissCross()

• Added Tensor mode to toSum, toAvg, toMin, toMax and oneToMany.

• Added alias toMean.

• Added dtype choice to toTensor.

0.3

Background stuff:

• Added mo tutorial

DL:

• Added FAnim()

• Added mo for stuff related to molecular dynamics

• Added graph(), digraph(), __call__()

• Huge revamp of schedule to make it more intuitive. There’s only 1 main schedule object now: Fn.

• Added k1lib.wraps(), k1lib.Object.__delitem__(), k1lib.Range.fromRange()

• Added k1lib.Every

Bioinfo cli:

• Removed notIn, as inSet does the job just as well

• Included cli import command in imports by default

• Modified toFloat() and modifier.toInt() so they can force weird values to 0.

• Moved existing toDict to toDictF, and created a new toDict.

0.2

Pretty minor update. I just want to push this out as soon as possible

Background stuff:

• Changed versioning system so that it makes more sense

• Added osic and mp tutorials

• Added more speed analysis in the cli tutorial

Bioinfo cli:

• Removed dirs and files parameters in ls, as isFile is good enough

• Disable module ctx by default, to improve performance

• Replaced nonEmptyStream to empty

0.1.13

Background stuff:

• Removed walrus operator library-wide, so that it can be used with Python < 3.8

Bioinfo cli:

• Added breakIf, isFile

• Replaced infiniteF with repeatF

• Replaced infiniteFrom with repeatFrom

• Replaced dereference with deref

• Replaced ctx.dereference with ctx.deref

• Modified applyMp to use dill to deserialize everything

• Modified to1Str so that it returns a string, instead of yielding

• Modified toStr so that it can accept an optional column

• Modified splitList so that it accepts *args

• Removed applySingle, but retain applyS

DL:

• Added class CaptureStdout

• Patched torch.Tensor with crissCross(), histBounds() and histScaled()

0.1.12

Background stuff:

• Removed docs’s generated fonts folder, as it takes 8MB and isn’t even used. Should now be sustainable for 100 versions on github pages.

Bioinfo cli:

• Removed wrapping __ror__ operator if __init__ is not defined, to make things performant.

• Added speed analysis in the cli tutorial.

• Added init.BaseCli.__call__(), consume, ctx.ctx(), ctx.f, peekF, expandE, utils.dereference.__invert__(), unique, notIn, sortF, toMax, toMin, union, ls(), instanceOf(), randomize

• Removed ctx.identity

• Added shortcuts to ctx.setC() and ctx.getC()

• Made headerIdx() context-aware

• Put ctx.Promise dereferencing at init.BaseCli.__ror__(), instead of dynamic patching at k1lib.bioinfo.cli construction time

• Get rid of stdout style inconsistencies. Previously, it looks like [1, 2] | stdout, now it looks like [1, 2] | stdout().

• Added multiprocessing capabilities with applyMp

• Added maxDepth option for dereference

• Added includeLast option to batched

• Added float("inf") option to bs parameter of batched

• Fixed permute to be able to take in generator for rows.

• Fixed stdout to be able to display non-iterable inputs

• Fixed dereference so that it will handle poking errors

• Fixed count so that it can deal with list. Used to throw unhashable type: ‘list’

• Added more flexibility with toFloat(), toInt() and sort

DL:

• Added classes AutoIncrement, Wrapper

• Added functions positionalEncode(), debounce()

• Fixed executeNb’s _globals doc-backref problem. Also clears plot automatically after executing a cell now.

0.1.11

Background stuff:

• Added testing section for unit tests (simple assert statements inside the notebooks)

Bioinfo cli:

• Added numpy/torch checks to dereference

• Renamed sample to peek

• Added infiniteFrom, joinStreamsRandom, batched, collate, splitList

• Added infiniteF

• Added toTensor

• Replaced infinite in favor of repeat

• Updated all cli tools to use super().__init__()

• Added ctx module, with Promise, enum, identity classes, and getC(), setC() methods

DL:

• Added k1lib.data.CyclicRandomSampler and k1lib.data.DatasetWithSampler

• Added highlight feature in callbacks.Callbacks.checkpointGraph()

• Added k1lib.viz.Carousel.pop(), k1lib.viz.Carousel.__getitem__()

• Added k1lib.Learner.sample()

• Disallow k1lib.executeNb() to execute lines that starts with “!”, also removed parameter catchErrors, as it’s quite useless

• Added k1lib.Object()

0.1.10

Background stuff:

• Added notebook regression tests

• Added tutorials automated building tool

• Added k1lib.ignoreWarnings

• Tutorials part moved to official docs (rather than .md files on github)

• Added covid tutorial

Bioinfo cli:

• Reintroduced permute, as rows is inadequate

• Added lengths, applySingle

• Exposed serial, oneToMany, manyToMany, and manyToManySpecific

DL:

• Added confusionMatrix.ConfusionMatrix and k1lib.viz.confusionMatrix()

• Added callbacks.Callbacks.checkpointGraph()

• Added torch.nn.modules.Module.paramsContext()

• Modified k1lib.Object to allow setting getdoc field

• Added k1lib.viz.Carousel.saveBytes(), k1lib.viz.Carousel.saveFile(), k1lib.viz.Carousel.saveGraphviz(), and handle different image formats much better now

0.1.9

Bioinfo cli:

• Added k1lib.Domain class

• Added __invert__ option for rows and columns by incorporating k1lib.Domain

• Sped up rows significantly

• Removed cli tool every as rows is good enough

• Added __invert__ option for head. As a result, removed nhead

• Added till() mechanism to grep cli tool

• Added gb cli module to deal with genbank file format

• Added tableFromList() as alias of listToTable()

• Renamed avg to toAvg

• Added toSum

DL:

• Added Callbacks tab completion to k1lib.Learner and Callbacks

• Refactored lossLandscape.LossLandscape to landscape.Landscape that can be used by any other callbacks, right now it’s Loss and Accuracy

• Added callbacks.Callback.pause() to help with Accuracy’s Landscape

0.1.8

• Fixed installation bug where readme.md would be absent, and setup.py won’t work at all.

Bioinfo cli:

• Huge revamp of how tables are treated. Before, row elements are in a single string, separated by a delimiter. Now each row is a full fledged list

• Added grepToTable, joinList, contains, table, stitch, listToTable, insertColumn cli tools

• Removed joinRows but still keep joinStreams as “joinRows” is not really obvious

• Removed permute as columns is pretty much identical

• Renamed inside to inSet

• Cut out nested functionality from grep to grepToTable

0.1.7

Interface with external modules:

• Swapped out register_backward_hook with register_full_backward_hook() in HookModule callback

• Added torch.nn.Module.preserveDevice() context manager

Callbacks:

• Changed Callbacks.suspend() to take in names only

• Added Callbacks.suspendClasses() to take in classes

• Changed Callbacks.suspendEvaluation() to Callbacks.suspendEval(), also added functionality for white and black lists

• Ripped out complex Callback.__getattr__() mechanism (that automatically searches the parent learner for missing attributes) to improve speed

• Added k1lib.callbacks.core callbacks CoreNormal and CoreRNN

• Added more plot decorators for MemoryProfiler, including backward-forward separator with their text

• Changed LossLandscape callback so that it follows the common flow structure. Before, it implements a whole new cbs(...) schema that essentially duplicated the loop inside of k1lib.Learner

Learner:

• Implemented grace stop handling for Learner. Before, if someone throws CancelRunException while executing a batch, then it will immediately jump to the end of the run. Changed it so that it will be caught in the batch loop, execute checkpoints cancelBatch and endBatch and rethrow it. Same thing with CancelEpochException

• Moved loss function from being a core functionality inside Learner to an external callback, so that adapting multiple loss functions are possible

• Added Learner.__call__()

Bioinfo cli:

• Added isValue, intersection, AA_, infinite, dereference cli tools

• Modified cat to be able to take in file name by pipe

• Added new joining operator “+” between init.BaseCli

Style changes:

• Changed Callbacks.learner to Callbacks.l

• Changed Callback.learner to Callbacks.l

• Removed k1lib.eqn from k1lib.imports’s namespace

0.1.6

I don’t really keep track of things 0.1.6 and before, so nothing here really.