k1lib package

Subpackages

• k1lib.bioinfo package
• k1lib.callbacks package

Submodules

• k1lib.data module
• k1lib.eqn module
• k1lib.format module
• k1lib.graphEqn module
• k1lib.imports module
• k1lib.nn module
• k1lib.schedule module
• k1lib.selector module
• k1lib.viz module
• k1lib.website module

Installation

Just do this:

pip install k1lib

Then in a notebook, do this:

from k1lib.imports import *
from k1lib.bioinfo.cli import *

This package has very few dependencies, and all of them are very commonly used

Repo is at https://github.com/157239n/k1lib/ btw.

Module contents

class k1lib.Learner

Bases: object

property model

Set this to change the model to run

property data

Set this to change the data (of type k1lib.data.Data) to run against.

property opt

Set this to change the optimizer. If you’re making your own optimizers, beware to follow the PyTorch’s style guide as there are callbacks that modifies optimizer internals while training like k1lib.schedule.ParamScheduler.

property cbs

The Callbacks object. Initialized to include all the common callbacks. You can set a new one if you want to.

property css

The css selector string. Set this to select other parts of the network. After setting, you can access the selector like this: l.selector

See also: ModuleSelector

property lossF

Set this to specify a loss function.

evaluate()

Function to visualize quickly how the network is doing. Undefined by default, just placed here as a convention, so you have to do something like this:

l = k1lib.Learner()
def evaluate(self):
    xbs, ybs, ys = self.Recorder.record(1, 3)
    plt.plot(torch.vstack(xbs), torch.vstack(ys))
l.evaluate = partial(evaluate(l))

__call__(xb, yb=None)

Executes just a small batch. Convenience method to query how the network is doing.

Parameters

• xb – x batch

• yb – y batch. If specified, return (y, loss), else return y alone

static load(fileName: Optional[str] = None)

Loads a Learner from a file. See also: save()

Parameters

fileName – if empty, then will prompt for file name

run(epochs: int, batches: Optional[int] = None)

Main run function.

Parameters

• epochs – number of epochs to run. 1 epoch is the length of the dataset

• batches – if set, then cancels the epoch after reaching the specified batch

save(fileName: Optional[str] = None)

Saves this Learner to file. See also: load()

Parameters

fileName – if empty, then will save as “learner-0.pth”, with 0 changeable to avoid conflicts. If resave this exact Learner, then use the old name generated before

class k1lib.Object

Bases: object

Convenience class that acts like defaultdict. You can use it like a normal object:

a = Object()
a.b = 3
print(a.b) # outputs "3"

__repr__() output is pretty nice too:

<class '__main__.Object'>, with attrs:
- b

You can instantiate it from a dict:

a = Object.fromDict({"b": 3, "c": 4})
print(a.c) # outputs "4"

And you can specify a default value, just like defaultdict:

a = Object().withAutoDeclare(lambda: [])
a.texts.extend(["factorio", "world of warcraft"])
print(a.texts[0]) # outputs "factorio"

Warning

Default values only work with variables that don’t start with an underscore “_”.

Treating it like defaultdict is okay too:

a = Object().withAutoDeclare(lambda: [])
a["movies"].append("dune")
print(a.movies[0]) # outputs "dune"

static fromDict(_dict: Dict[str, Any])

Creates an object with attributes from a dictionary

property state

Essentially __dict__, but only outputs the fields you defined. If your framework intentionally set some attributes, those will be reported too, so beware

withAutoDeclare(defaultValueGenerator)

Sets this Object up so that if a field doesn’t exist, it will automatically create it with a default value.

withRepr(_repr: str)

Specify output of __repr__(). Legacy code. You can just monkey patch it instead.

class k1lib.Range(start=0, stop=None)

Bases: object

A range of numbers. It’s just 2 numbers really: start and stop

This is essentially a convenience class to provide a nice, clean abstraction and to eliminate errors. You can transform values:

Range(10, 20).toUnit(13) # returns 0.3
Range(10, 20).fromUnit(0.3) # returns 13
Range(10, 20).toRange(Range(20, 10), 13) # returns 17

You can also do random math operations on it:

(Range(10, 20) * 2 + 3) == Range(23, 43) # returns True
Range(10, 20) == ~Range(20, 10) # returns True

__getitem__(index)

0 for start, 1 for stop

You can also pass in a slice object, in which case, a range subset will be returned. Code kinda looks like this:

range(start, stop)[index]

__init__(start=0, stop=None)

Creates a new Range.

There are different __init__ functions for many situations:

• Range(2, 11.1): create range [2, 11.1]

• Range(15.2): creates range [0, 15.2]

• Range(Range(2, 3)): create range [2, 3]. This serves as sort of a catch-all

• Range(slice(2, 5, 2)): creates range [2, 5]. Can also be a range

• Range(slice(2, -1), 10): creates range [2, 9]

• Range([1, 2, 7, 5]): creates range [1, 5]. Can also be a tuple

fixOrder() → k1lib._basics.Range

If start greater than stop, switch the 2, else do nothing

intIter(step: int = 1) → Iterator[int]

Returns integers within this Range

toUnit(x)

Converts x from current range to [0, 1] range. Example:

r = Range(2, 10)
r.toUnit(5) # will return 0.375, as that is (5-2)/(10-2)

You can actually pass in a lot in place of x:

r = Range(0, 10)
r.toUnit([5, 3, 6]) # will be [0.5, 0.3, 0.6]. Can also be a tuple
r.toUnit(slice(5, 6)) # will be slice(0.5, 0.6). Can also be a range, or Range

Note

In the last case, if start is None, it gets defaulted to 0, and if end is None, it gets defaulted to 1

fromUnit(x)

Converts x from [0, 1] range to this range. Example:

r = Range(0, 10)
r.fromUnit(0.3) # will return 3

x can be a lot of things, see toUnit() for more

toRange(_range: k1lib._basics.Range, x)

Converts x from current range to another range. Example:

r = Range(0, 10)
r.toRange(Range(0, 100), 6) # will return 60

x can be a lot of things, see toUnit() for more.

property range_

Returns a range object with start and stop values rounded off

property slice_

Returns a slice object with start and stop values rounded off

static proportionalSlice(r1, r2, r1Slice: slice) → Tuple[k1lib._basics.Range, k1lib._basics.Range]

Slices r1 and r2 proportionally. Best to explain using an example. Let’s say you have 2 arrays created from a time-dependent procedure like this:

a = []; b = []
for t in range(100):
    if t % 3 == 0: a.append(t)
    if t % 5 == 0: b.append(1 - t)
len(a), len(b) # returns (34, 20)

a and b are of different lengths, but you want to plot both from 30% mark to 50% mark (for a, it’s elements 10 -> 17, for b it’s 6 -> 10), as they are time-dependent. As you can probably tell, to get the indicies 10, 17, 6, 10 is messy. So, you can do something like this instead:

r1, r2 = Range.proportionalSlice(Range(len(a)), Range(len(b)), slice(10, 17))

This will return the Ranges [10, 17] and [5.88, 10]

Then, you can plot both of them side by side like this:

fig, axes = plt.subplots(ncols=2)
axes[0].plot(r1.range_, a[r1.slice_])
axes[1].plot(r2.range_, a[r2.slice_])

bound(rs: Union[range, slice]) → Union[range, slice]

If input range|slice’s stop and start is missing, then use this range’s start and stop instead.

copy()

__invert__()

class k1lib.Domain(*ranges, dontCheck: bool = False)

Bases: object

__init__(*ranges, dontCheck: bool = False)

Creates a new domain.

Parameters

• ranges – each element is a Range, although any format will be fine as this selects for that

• dontCheck – don’t sanitize inputs, intended to boost perf internally only

A domain is just an array of Range that represents what intervals on the real number line is chosen. Some examples:

inf = float("inf") # shorthand for infinity
Domain([5, 7.5], [2, 3]) # represents "[2, 3) U [5, 7.5)"
Domain([2, 3.2], [3, 8]) # represents "[2, 8)" as overlaps are merged
-Domain([2, 3]) # represents "(-inf, 2) U [3, inf)", so essentially R - d, with R being the set of real numbers
-Domain([-inf, 3]) # represents "[3, inf)"
Domain.fromInts(2, 3, 6) # represents "[2, 4) U [6, 7)"

You can also do arithmetic on them, and check “in” oeprator:

Domain([2, 3]) + Domain([4, 5]) # represents "[2, 3) U [4, 5)"
Domain([2, 3]) + Domain([2.9, 5]) # represents "[2, 5)", also merges overlaps
3 in Domain([2, 3]) # returns False
2 in Domain([2, 3]) # returns True

static fromInts(*ints: List[int])

Returns a new Domain which has ranges [i, i+1] for each int given.

copy()

intIter(step: int = 1, start: int = 0)

Yields ints in all ranges of this domain. If first range’s domain is \((-\inf, a)\), then starts at the specified integer

exception k1lib.CancelRunException

Used in core training loop, to skip the run entirely

exception k1lib.CancelEpochException

Used in core training loop, to skip to next epoch

exception k1lib.CancelBatchException

Used in core training loop, to skip to next batch

k1lib.textToHtml(text: str) → str

Transform a string so that it looks the same on browsers as in print()

k1lib.clearLine()

Prints a character that clears the current line

k1lib.tab(text: Union[list, str]) → Union[list, str]

Adds a tab before each line. str in, str out. List in, list out

k1lib.isNumeric(x) → bool

Returns whether object is actually a number

k1lib.close(a, b)

Returns whether 2 values are really close to each other

k1lib.stats(x) → Tuple[float, float]

Returns the mean and standard deviation of the input

k1lib.patch(_class: type, name: Optional[str] = None, docs: Optional[Union[str, Any]] = None, static=False)

Patches a function to a class/object.

Parameters

• _class – object to patch function. Can also be a type

• name – name of patched function, if different from current

• docs – docs of patched function. Can be object with __doc__ attr

• static – whether to wrap this inside staticmethod or not

Intended to be used like this:

class A:
    def methA(self):
        return "inside methA"

@patch(A)
def methB(self):
    return "inside methB"

a = A()
print(a.methB()) # will print "inside methB"

k1lib.squeeze(_list: Union[list, tuple, torch.Tensor, Any], hard=False)

If list only has 1 element, rethrn that element, else return original list

Parameters

hard – If True, then if list/tuple, filters out None, and takes the first element out even if that list/tuple has more than 1 element

k1lib.raiseEx(ex: Exception)

Raises a specific exception. May be useful in lambdas

k1lib.smooth(arr: List[float], consecutives: int = 5) → List[float]

Smoothes out array, so that y values are averages of the neighbors

k1lib.numDigits(num) → int

Get the number of digits/characters of this number/object

k1lib.limitLines(s: str, limit: int = 10) → str

If input string is too long, truncates it and adds ellipsis

k1lib.limitChars(s: str, limit: int = 50)

If input string is too long, truncates to first limit characters of the first line

k1lib.showLog(loggerName: str = '', level: int = 10)

Prints out logs of a particular logger at a particular level

k1lib.beep()

Plays a beeping sound, may be useful as notification for long-running tasks

k1lib.executeNb(fileName: str, _globals: dict = {})

Executes a specified IPython notebook. Can make all variables defined in the notebook appear in the __main__ context by passing globals() in

k1lib.dontWrap()

Don’t wrap horizontally when in a notebook

k1lib.polyfit(x: List[float], y: List[float], deg: int = 6) → Callable[[float], float]

Returns a function that approximate \(f(x) = y\).

Parameters

deg – degree of the polynomial of the returned function

k1lib.derivative(f: Callable[[float], float], delta: float = 1e-06) → Callable[[float], float]

Returns the derivative of a function

k1lib.optimize(f: Callable[[float], float], v: float = 1, threshold: float = 1e-06) → float

Given \(f(x) = 0\), solves for x, using initial value v

k1lib.inverse(f: Callable[[float], float]) → Callable[[float], float]

Returns the inverse of a function. The inverse function takes a long time to run, so don’t use this where you need lots of speed.

k1lib.integrate(f: Callable[[float], float], _range: k1lib._basics.Range) → float

Integrates a function over a range