See also my home page. I'm a career quant, applied mathematician, open-source developer, entrepreneur and father of three girls.
| Site | Topic |
|---|---|
| schur.microprediction.org | Schur complementary portfolios — a unification of hierarchical and optimisation-based portfolio construction via block-matrix inversion. Now part of skfolio. |
| thurstone.microprediction.org | Thurstone Class V models and the fast ability transform — multi-entrant contest probabilities and inverse calibration, with eight interactive in-browser demos and a JS port of the Python analytics. |
| Site | Topic |
|---|---|
| humpday.microprediction.org | All the greatest derivative-free (black box) optimization methods in one place in pure Python and Javascript with no package dependencies. Comparison and selection tooling, and interactive demos. |
| skaters.microprediction.org | Fast and surprisingly reliable online univariate time series algorithms providing distributional predictions. Zero dependencies, runs in Pyodide. |
| Page | Content |
|---|---|
| book.microprediction.org | Microprediction — the book on splintering data science into small algorithmic agents. (Audio book too) |
| microprediction.medium.com | Mostly ong-form posts on portfolio construction, contests, microprediction, time series, and quantitative finance. |
| microprediction/home | Papers - and some short musings |
If you were looking for the cult article Is Facebook's Prophet the Time-Series Messiah or Just a Very Naughty Boy?, it moved here.
- Quant finance. Portfolio and ensemble construction (e.g. paper and blog where I unified the two sides of portfolio theory - more reading if you wish, and here's a broader papers list on the topic. Also trading, microstructure (past work here and there and there but mostly private).
- Collective Intelligence
- Sports analytics
- Mathematics in general
There's a custom GPT you can ask. Short version is that a few years ago I wrote a book predicting that data science would splinter into little agents. I've long been a believer in engineering pipelines that anyone else can improve without asking permission, and in the eventual inversion of control between humans and machine in the "microprediction domain" (frequently repeated quantitative tasks).
I'm realized, with the arrival of LLMs, that this applies to judgemental prediction (less frequently or never repeated) also. It seems I have more faith in small markets than most, noting the important caveat made clear in the Indispensable Markets Hypothesis paper that markets can be indispensible yet not perfectly efficient.
The book was a meditation on the power of mini-markets and algorithmic statistical agents - a thesis that went from unlikely to almost self-evident as LLMs arrived. It predates phrases like "DeAI" and "Info Finance" (Buterin) not to mention the general explosion of interest in prediction markets ... but despite this shift in the zietgeist the ideas have a long way to go as far as seeping into general software engineering consciousness in concerned (judging by this market anyway).
- Firstdown repo contains analysis aspiring to ruin great game of football. See Wilmott paper and for heaven's sake, don't stretch out for the first down. That's obviously nuts.
- manifoldbot - A bot that uses LLMs to trade on manifold prediction markets.
- randomcov - A set of quirky correlation and covariance matrix generators (I'd love your ideas).
- embarrassingly - A speculative approach to robust optimization that sends impure objective functions to optimizers.
- pandemic - Ornstein-Uhlenbeck epidemic simulation (related paper)
- momentum - My most personally re-used mini package ... for incremental mean, var, skew, kurtosis.
- muid - Memorable Unique Identifiers ... try to figure out how that can't be an oxymoron.
- timeseries-notebooks - Lots of examples of using open source timeseries packages.
- correlationbounds - Mini package for conf bounds
- building_an_open_ai_network - Book related.
- recalibrate - Utils related to Platt scaling etc.
The client for microprediction: a platform sponsored by Intech Investments that collected a billion predictions. It has entered a trisoloran dehydrated state but maybe will be revived at a future date, after one of my three hundred ChatGPT generated scientific grant proposals is successful. Some relics remain:
- The muid identifier package is explained in this video.
- There are other rats and mice like getjson, runthis and momentum that still work, I think.
Not so fresh:
- The "/skaters" provided canonical, single-line of code access to functionality drawn from packages like river, pydlm, tbats, pmdarima, statsmodels.tsa, neuralprophet, Facebook Prophet, Uber's orbit, Facebook's greykite and more. Though now that superceded by skaters, actually.
- Choices were sometimes advised by Elo ratings, but anyone can do what they want.
- It's not too hard to use my HumpDay package for offline meta-param tweaking, et cetera. Now fully revamped.
- The precise package for online ensembling still works, but could use a coat of paint.