Here’s a breakdown of some of the software projects I have worked on with some reflections on the successes, challenges and failures along the way.
March 2021 - present
Incremental writing is a method of writing in which ideas are written down and assembled incrementally over months and years. I recommend watching this video I made if you are curious about incremental writing.
Overall, writing this plugin was surprisingly easy. Obsidian has really good support for developing plugins because it was built from the beginning to be extensible like VSCode rather than having a plugin framework clumsily retrofitted to it.
The plugin API was still in beta so there were no comments or documentation other than the TypeScript type declarations, but there were tons of projects on GitHub that I could read to find out how to do common operations.
One of the best parts about this plugin is that it stores its data transparently as plain text in a markdown table. This allows users to view and edit their incremental writing queues from within Obsidian and also allows my plugin to integrate alongside other plugins, such as the Advanced Markdown Tables Plugin. I also added a number of commands to allow users to bulk add existing notes to an incremental writing queue which made it easy for long-term users of Obsidian with large vaults of notes to get started.
October 2020 - August 2021
Check it out on GitHub!
I used test-driven development with the XUnit testing framework and to make sure that the code for converting between C# DateTimes and the strange time format used by SuperMemo’s SleepChart was correct (the start and end times of each sleep block were represented by a float where the integral part represents the number of days since Jan 1st 2000 and the fractional part represents the percentage of the day that has passed to the nearest minute).
Another developer used this library to create a library for automatically adding sleep data from Fitbit fitness trackers to SuperMemo's SleepChart file.
October 2020 - August 2021
I reverse engineered part of the SuperMemo source code to enable SuperMemoAssistant plugin developers to interact with sound components.
Since SuperMemo is not open source, calling SuperMemo native functions from SuperMemoAssistant (open source plugin framework for SuperMemo, not affiliated with the SuperMemo developers) requires reverse engineering the SuperMemo source code with IDA and the Interactive Delphi Reconstructor (IDR).
While working on my incremental media player project, I wanted to add a feature that would allow a SuperMemoAssistant plugin to be able to read and write to the audio file stored inside a sound component. To do this, I loaded the SuperMemo binary in IDR and searched for functions related to sound components. By inspecting the disassembled code I was able to work out the functions I needed and the parameters required. I used the CheatEngine debugger to make sure I had found the correct functions, and then used IDA to create code signatures for the functions so SuperMemoAssistant could scan for them at runtime. Next I just needed to make some small edits to the SuperMemoAssistant source code to make the new functions callable from my plugin project.
This was a pretty fun project because it gave me a lot of motivation to dive into reverse engineering with IDA. Without a meaningful project like this, I think it would have been very difficult to persist and overcome the confusion stemming from the complexity of IDA and reading assembly. But having completed the project, I gained more confidence with basic reverse engineering and I am looking forward to developing these skills further in the future.
January 2021 - present
Lua, C#, Python
Check it out on GitHub
The incremental media player is a cross-platform application for incrementally consuming audio and video content. The software basically helps you manage queues of prioritized audio and video tracks stored on your local disk or on YouTube. The program helps you gradually convert passive audio and video files into active recall multimedia flashcards. The player itself is implemented as an mpv Lua script. I also wrote a SuperMemoAssistant plugin and a cross-platform background service in C# to integrate the mpv script with SuperMemo.
The idea of creating a program that could improve the effectiveness of learning from audio and visual content was one of the core motivations for me to start programming back in mid-2019. While we know from the scientific literature that active recall is much more effective than passive review for learning, and while this understanding has been applied successfully in spaced repetition systems for text-based content, there are no systems that implement this effectively for audio and video.
The incremental media player evolved out of an earlier project called AudioAssistant. Back in May 2019 I started learning Linux and bash scripting so I could create an language learning audio player with a Raspberry Pi. I used a USB battery pack as a power supply and a USB numpad keyboard as a controller so I could record sentences and words from Chinese lectures and podcasts while commuting to university. At this stage, while I was able to extract sections of audio, I couldn't yet export them into a spaced repetition system or create flashcards.
Later I rewrote the program in Python with a proper sqlite database and added the ability to download YouTube videos, create “audio cloze” flashcards using a bluetooth remote and export the flashcards into SuperMemo. Audio cloze flashcards are flashcards where the question side is a sentence where a word or phrase has been "censored" with a beep sound. When you are presented with the question side, you have to try and recall which word or phrase has been covered by the beep. I would often ride around on my bike taking extracts from lectures and making audio cloze deletions from Chinese podcasts that I could later import into SuperMemo to review over time (I must have looked ridiculous). These early prototypes showed me that the core of the idea was solid, but the program was pretty awkward to use, only supported audio and was tied to linux.
The current version of the software is my third or fourth attempt at creating the program that I first envisioned over two and a half years ago! I am glad to have finally reached the point where I can use it comfortably for both audio and video content, use it both as a standalone app or integrated into SuperMemo and use it cross platform.
I ran into a number of issues related to the programming language and environment in this project.
Thankfully, I was able to read the scripts in the enormous collection of mpv user scripts to gain a better understanding of the mpv API. Two scripts that really helped me were mpvacious and immersive which are both scripts to help Japanese language learners.
July 2021 - present
Check it out on GitHub!
The incremental media player background service uses JSON-RPC to coordinate communication between the SuperMemo incremental media player plugin and mpv. This is useful because many SuperMemo users run SuperMemo inside a virtual machine. The background service allows you to send messages from the SuperMemo plugin running inside a Windows virtual machine to mpv running on a linux host machine.
May 2020 - August 2020
Check it out on GitHub
A blog that I wrote in Python using the Flask web framework. Its defining feature was the use of a Neo4j graph database to store information about articles, implement a commenting system and add associative tagging relationships between articles.
I was interested in NoSQL databases at the time and I decided to use one for my blog. Neo4j interested me in particular because graphs seemed to me to be better at representing associative relationships. I used the graph database to implement a commenting system from scratch and create a tagging system for articles. This was a good learning experience because there wasn’t much information out there about building a blog using a graph database like Neo4j, so I was forced to work things out myself. I used the project for my blog for around 6 months, but I eventually found it was missing some quality of life features. It was inconvenient to add or edit articles, there was no version control and it was more expensive to host because it was using Neo4j which is pretty resource hungry. Eventually I switched to using WikiJS, but I would definitely like to write my own blog from scratch again in the future rather than relying on a prepackaged tool.
April 2020 - June 2020
Check it out on GitHub
This plugin allows you to easily add hints to cloze flashcards in SuperMemo. It remembers the history of hints you added in the past, allowing you to use a dropdown menu to select hints and increase the speed of creating cloze deletion flashcards.
One conclusion I reached after reflecting on this plugin and the other software I have written so far is that when you want to create something that others will use, there is no "premium for complexity" - there is often no benefit to the user from adding complex features (though you could argue that there is a learning benefit for the developer). So while the plugin wasn’t exceptionally clever or difficult to make, it improved users’ workflows significantly and became quite popular.
The mouseover popup plugin allows you to quickly preview the content of urls for different sites by hovering over links in SuperMemo HTML components. The plugin publishes itself as a SuperMemoAssistant Service and handles popup sizing and positioning, while the HTML content for the popup is provided by “content provider” plugins (eg. Mouseover Wiki and Mouseover Guru) which register themselves with this plugin through SuperMemoAssistant's service locator.
I started off by creating a popup window to preview Wikipedia articles. It worked in a similar way to the popup windows on Wikipedia where you mouseover a link and it shows you a summary of the article and a picture if available. Later I realized I could extend the approach to other websites that are popular amongst SuperMemo users, like the SuperMemoGuru website. Finally I came up with the idea of having one main plugin that would handle the styling and positioning of the popup, and some "content provider" plugins that provide code to get the content from different sites.
I think this was one of the best plugins I wrote for improving users' workflows because it allows you to quickly preview an article's content and check whether it is interesting to you before adding it to your queue. When you are reading a dense technical scientific article it's also useful to be able to quickly pop open definitions of unknown concepts. And if you found that the preview content was enough to answer any questions you had about the concept, I added the ability to extract the content directly from the popup window into your collection, or open the full article in your browser.
More on these soon...