Understanding and processing natural language is a complex task. Embeddings are a powerful tool in todays ML/AI discussions to address this problem.
<h2>Mental model</h2>
In my mental model, embeddings are used to encode words into meaning. Embeddings are encodings in n-dimensional vector space where every parameter is a signed a specific meaning in the object space.
I can't really imagine n-dimensional vectors but for an embedding system with two vectors, I like to visualize it like this:
<div class="remark-highlight"><pre class="language-unknown"><code class="language-unknown">terrain (y-axis)
|
|
| o (mountain bike)
|
| o (road bike)
|
|
| o (horse)
|
| o (car)
|
|
+------------------------------- speed (x-axis)</code></pre></div>
In this example, each point represents a mode of transportation. Items with similar speed and terrain capabilities are placed closer together. For example, a mountain bike and a road bike are closer to each other because they have similar speeds and are both used on different terrains, while a car is further away since it operates at higher speeds on more paved terrains.
There are two super-powers from being able to map natural language onto an n dimensional vector space:
<ul>
<li>We can use relatively simple vector algorithms to quickly find relevant words. Algorithms like <a href="https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm">kNN</a> are easy to be optimized with spacial maps so lookup is fast. In our example, we can see that a mountain bike is closer to a road bike than it is to a horse.</li>
<li>Embeddings compress insanely well. It's entirely possible to compress pages of written language into a vector size of a few hundred vectors.</li>
</ul>
There are also some downsides, though:
<ul>
<li>Training an embedding system is like training any other ML system: It's a complex and expensive task. And even with a fully trained model, you will need a lot of compute power to convert a piece of text into embeddings. That's why it's currently common to outsource embeddings generation to third-party APIs like OpenAI.</li>
<li>When a model is trained, we don't really know what meaning each value of the vector is assigned to. This makes it hard to understand why a specific pair of text is considered close to each other or not.</li>
</ul>
<h2>How are embeddings used?</h2>
Because of the fast way to find related vector pairs, embeddings are often used to answer search problems. In the current AI discourse, embeddings are used to quickly and efficiently find relevant context in order to put it into the prompt of a large-language model (LLM).
Imagine an AI system to ask questions about your code documentation. We can, ahead of time, create embeddings for every paragraph in the documentation and store it in a vector database (a database that can hold the embedding vectors for us and has some index to efficiently query it. Like the <a href="https://github.com/pgvector/pgvector">pgvector</a> extension for Postgres).
Now, when the user enters a question in natural text into our AI system, we can find the most relevant paragraphs into our documentation. We usually want to pick as many as we can fit into the prompt of the LLM. The LLM now has an understanding of your documentation when answering the question. Fascinating!
One important thing from the paragraphs above is that we have to split the documentation into meaningful chunks. Since we can only retrieve similarity in windows of this chunk site, you can easily see how important it is to pick a proper system.
<h2>Some links I want to remember</h2>
<ul>
<li><a href="https://huggingface.co/blog/getting-started-with-embeddings">Getting Started With Embeddings</a></li>
<li><a href="https://huggingface.co/blog/mteb">MTEB: Massive Text Embedding Benchmark</a></li>
<li><a href="https://huggingface.co/spaces/mteb/leaderboard">MTEB Leaderboard</a></li>
</ul>

Embeddings

When building an AI code assistant like <a href="https://about.sourcegraph.com/cody">Cody</a>, you often find yourself in need for a quick way to rank the similarity of two blocks of text relative to each other.
This is where <a href="https://en.wikipedia.org/wiki/Jaccard_index">Jaccard similarity</a> is helpful. Jaccard similarity is a metric to find similarity between two sets of data, typically used for comparing text data.
The equation for Jaccard distance is:
<div class="remark-highlight"><pre class="language-unknown"><code class="language-unknown">J(A, B) = |A ∩ B| / |A ∪ B|</code></pre></div>
Here's a simple example in TypeScript to calculate the Jaccard distance between two sentences:
<div class="remark-highlight"><pre class="language-typescript"><code class="language-typescript">function jaccardDistance(sentence1: string, sentence2: string): number {
 const set1 = new Set(sentence1.split(' '));
 const set2 = new Set(sentence2.split(' '));
 
 const intersection = new Set([...set1].filter(x => set2.has(x))).size;
 const union = new Set([...set1, ...set2]).size;
 
 return intersection / union;
}
</code></pre></div>
The Jaccard similarity comes in handy for finding similar text, like detecting plagiarism or comparing documents. It might not be the most advanced method for text comparison, but it's simple and fast, making it great for many applications.
You can even do some pre-processing on the text to increase its accuracy. The above example is naively comparing words but you could use <a href="https://en.wikipedia.org/wiki/Stemming">stemming</a> to make it easier to compare words with the same meaning.
Why not give it a try with some sample texts and see how it works for you?

Jaccard Similarity

A lot of companies love ClickHouse for it's great performance as LDAP system, incredible compression, and its ease of deployment.
This is a list of features that are perhaps less known about but also quite powerful:
<ul>
<li><a href="https://clickhouse.com/docs/en/sql-reference/statements/alter/column#materialize-column">Materialized Columns</a>: This feature allows you to add new columns (which, given the fact that ClickHouse is column based, is very cheap) that derive the data from existing fields. You can imagine a generic log table with a json text field and you want to extract a specific field from the JSON into a custom column. With this feature, you can start your log table in an unstructured form and gradually migrate to a schema over time.
Check out <a href="https://www.youtube.com/watch?v=pZkKsfr8n3M">this video from Altinity</a> for more informations.</li>
<li><a href="https://clickhouse.com/docs/en/guides/developer/ttl">TTL</a>: ClickHouse allows you to define how long data inside tables and even specific columns is being retained. I think this is an amazing feature for sensitive data, ensuring the data is only retained as long as it is necessary for analytical purposes.</li>
<li><a href="https://clickhouse.com/docs/en/interfaces/schema-inference#json-formats">Automatic schema inference from JSON</a>: Granted, this is a very niche feature but I assume it could be amazing in combination with materialized columns to gradually and automatically convert an untyped JSON payload into a typed column structure.</li>
<li><a href="https://clickhouse.com/docs/en/sql-reference/statements/create/quota">Quotas</a>: A feature to allow you to limit resources in a multi-tenant ClickHouse deployment. You don't care about it until you absolutely need it, and here you just get it for free.</li>
</ul>

Niche ClickHouse Features

This is a list of non-trivial problems I have come across in the front-end community over the past few years. It mostly exists as a personal reminder. <a href="https://twitter.com/PhilippSpiess">Let me know</a> if you have something that should be added here!
<ul>
<li>
Accessibility: Making web apps accessible because it’s so easy with the Web fundamentals to break things.
</li>
<li>
Bundle coverage: A lot of the code that is shipped in today’s bundles (CSS, JS, etc.) is not needed for the first render.
</li>
<li>
Data waterfalls: Waterfalls is a term used to describe serial loading dependencies. E.g. to render a list of users we first download the list component, wait for it to load, then the list component makes a request to an API, waits for it to load, etc.
This is the number one issue that I have seen causing websites to be slow. It often manifests itself as a myriad of uncontrolled loading states that sometimes end in another loading state popping up. 😵‍💫
An interesting observation: Most applications could be able to resolve all the data needed for the initial page load on the server side based on the URL, params, and authentication tokens.
</li>
<li>
Overfetching/underfetching: A client should only ever get as many data as it needs to render. Anything more is not needed and everything less is not enough.
</li>
<li>
Interaction delays: If I interact with an item on the website, it should show an instant response.
</li>
<li>
Layout shifting: Once the first elements on a website are shown, you don't want to have them jump around. This only causes confusion to the user and results in a poor experience.
</li>
<li>
Developer experience: I think this term is often overused but the current way we write apps is definitely not the end game. We need constant improvements to the tooling because the cost of building is still way too high.
</li>
<li>
Hydration: When you have a server-rendered HTML page and it needs to become interactive ASAP.
</li>
<li>
Reliability: A meta-term that is all about knowing if your app functions. There are many vectors this can be done, however easy mechanisms also have a weak correlation to actual user problems.
<ul>
<li>Error handling/tracking. Unfortunately knowing that an error was thrown does not often relate to a broken website—Is the error recoverable by a user or is it crashing the whole page?</li>
<li>Browser crashes/freezes/loops</li>
<li>Uptime</li>
<li>Tracking core interaction uptime</li>
</ul>
</li>
<li>
Performance: This is a very convoluted term that justifies it’s own note. Until then, here are some thoughts:
<ul>
<li>Measuring page loads is usually simpler than measuring interactions (navigations, popups).</li>
<li>You can measure performance in a lab but this will only ever be a proxy to the real-user performance metrics (aka RUM—Real User Monitoring).</li>
</ul>
</li>
<li>
Efficiency: Another meta-term for a few lower-level optimzations that rarely cause problems (but when they do, they are noticeable):
<ul>
<li>Memory usage</li>
<li>Heavy CPU usage</li>
<li>Uncontrolled scheduled tasks (e.g. <code>setInterval</code>)</li>
<li>Browser rendering issues</li>
<li>Network/IO overhead</li>
</ul>
</li>
<li>
Testing: While software testing is hard in every environment, front-end has a couple of special complexity with things like visual regression testing, accessibility testing, the reliance on a backend API, or the flakiness of browser instrumentation.
</li>
<li>
Observability: Knowing how your app functions is a complex problem in itself. Traditionally, the web platform was very limited in how much data we have available with a big focus on analytics. Lately there have been a few amazing developments though:
<ul>
<li><a href="https://github.com/tc39/proposal-async-context">Async Context</a> is a recent proposal to bring zones.js to the platform which will allow for easer tracing APIs in the future.</li>
<li><a href="https://github.com/WICG/js-self-profiling">JavaScript Self-Profiling API</a> brings process sampling features to the web platform.</li>
</ul>
</li>
</ul>

Modern Front-End Problems

<ul>
<li>When working on complex systems, there are often more than one test parameter that changes. For example when working on a component for your design system, you need to test across all these different parameters:
<ul>
<li>Responsive Breakpoints</li>
<li>Dark/Light Mode</li>
<li>Accesibility options (screen reader, mouse-only, keyboard-only)</li>
<li>Different browser versions</li>
</ul>
</li>
<li>A good abstraction to help cope with this increasing complexity is by thinking of combinational matrixes.
<ul>
<li>In code, this can be multi dimensional arrays or nested loops. <a href="https://jestjs.io/docs/api#testeachtablename-fn-timeout">Jest even has a feature for it</a>.</li>
<li>In PR descriptions, it's helpful to draw a table to make sure you test all possible combinations throghoutly.</li>
</ul>
</li>
<li>Increasing the dimensions can make a full test run very costly though, so you can’t just blindly add more.</li>
<li>A very common pattern where this is userful is for manual pull request descriptions where you want to provide proof that you have exhaustively tested a new feature. A two-dimensional table might be used to show screenshots of e.g. different browsers and different responsive breakpoints.</li>
</ul>

Test Matrixes

<ul>
<li><input type='checkbox' disabled /> Set up the <a href="https://github.com/patrick-zippenfenig/us-with-german-umlauts#:~:text=Mac%20U.S.%20keyboard%20layout%20with%20German%20Umlauts,-This%20layout%20is&#x26;text=The%20layout%20is%20a%20modified,option%20%2B%20u%20%3D%3E%20%C3%BC">US-German Keyboard layout</a></li>
<li><input type='checkbox' disabled /> Map <code>tab</code> key to <code>escape</code> in Keyboards settings</li>
<li><input type='checkbox' disabled /> Connect iCloud and enable Find My Mac</li>
<li><input type='checkbox' disabled /> Set up 1Password</li>
<li><input type='checkbox' disabled /> Connect Google account for the Calendar integration</li>
<li><input type='checkbox' disabled /> Homebrew</li>
<li><input type='checkbox' disabled /> Oh My Zsh
<ul>
<li><code>git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions</code></li>
<li><code>git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting</code></li>
</ul>
</li>
<li><input type='checkbox' disabled /> asdf</li>
<li><input type='checkbox' disabled /> Warp Terminal</li>
<li><input type='checkbox' disabled /> Raycast</li>
<li><input type='checkbox' disabled /> Discord</li>
<li><input type='checkbox' disabled /> Chrome
<ul>
<li><input type='checkbox' disabled /> Set up two profiles</li>
<li><input type='checkbox' disabled /> Change the UI theme of one of the profiles (Pref. work to blue)</li>
</ul>
</li>
<li><input type='checkbox' disabled /> Firefox</li>
<li><input type='checkbox' disabled /> Spotify</li>
<li><input type='checkbox' disabled /> Flow (new and being tried out, an app for the Pomodoro technique)</li>
<li><input type='checkbox' disabled /> Hidden Bar (tbd, an app to clean up the menu bar)</li>
<li><input type='checkbox' disabled /> VS Code</li>
<li><input type='checkbox' disabled /> Sync <a href="https://github.com/philipp-spiess/dotfiles">dotfiles</a></li>
<li><input type='checkbox' disabled /> Set up SSH keys for GitHub and remote servers</li>
<li><input type='checkbox' disabled /> Microsoft To Do</li>
<li><input type='checkbox' disabled /> Obsidian</li>
<li><input type='checkbox' disabled /> Paprika Recipe Manager</li>
<li><input type='checkbox' disabled /> Rectangle</li>
<li><input type='checkbox' disabled /> MonitorControl</li>
<li><input type='checkbox' disabled /> Telegram</li>
<li><input type='checkbox' disabled /> WhatsApp</li>
<li><input type='checkbox' disabled /> Things</li>
<li><input type='checkbox' disabled /> The Unarchiver</li>
<li><input type='checkbox' disabled /> Work Related
<ul>
<li><input type='checkbox' disabled /> Slack</li>
<li><input type='checkbox' disabled /> Loom</li>
<li><input type='checkbox' disabled /> Carbon Black</li>
<li><input type='checkbox' disabled /> Dev Environment</li>
<li><input type='checkbox' disabled /> Zoom</li>
</ul>
</li>
</ul>

New Mac Checklist

<ul>
<li>Being not constraint to a specific domain is one of React's killer features.</li>
<li>In React DOM, you can always fall back to creating custom <code>&#x3C;div></code> elements with your own styles if something doesn't work.
<ul>
<li>There's a maintainance overhead by doing so but it also unblocks so many use cases.</li>
<li>Even at the new Facebook.com, custom containers with with the full fallback to the browser were very common.</li>
</ul>
</li>
<li>Every good design systems that I have seen embraces this and allows their users to forward e.g. custom class names to their inner working components.</li>
</ul>

React’s Escape Hatch

A collection of my random thoughts. This is a place for me to write down raw ideas, so don’t expect a high bar for quality.
Inspired by Josh Branchaud’s <a href="https://github.com/jbranchaud/til">til</a>, Lee Byron’s <a href="https://leebyron.com/til/">til</a>, and Shawn Wang’s <a href="https://www.swyx.io/learn-in-public/">Learn In Public</a>.

Notes

Generative AI, whether for code, text, images, or other use cases, appears as a magic black box to many users. Users typically navigate to a website, install an app, or set up an extension and start seeing the results of the AI tool. But, have you ever wondered what goes into this magic black box or how it really works?

The Lifecycle of a Code AI Completion

Building a Schema Mapper in TypeScript

Scheduling in React

Electron is a powerful open source framework that allows development of cross-platform desktop GUI applications with JavaScript, HTML, and CSS. In the past few years, it has been adapted by major companies in the tech industry including GitHub, Facebook, Microsoft, and Slack.

Running Native Code in Electron and the Case for WebAssembly

In late 2016, we released the first version of our Web SDK, which relies on a server component to render documents. Only a few months later, we released an updated version of PSPDFKit for Web — one that doesn’t require a server component and instead uses WebAssembly (Wasm, WA) to render documents directly in the browser. This was a big achievement for us as a company, and being able to provide a browser-only solution that doesn’t require any backend infrastructure lowers the barrier to entry for our customers.

A Real-World WebAssembly Benchmark

In March 2017, the WebAssembly Community Group reached a consensus on the initial (MVP) binary format, JavaScript API, and reference interpreter. This exciting new technology is already shipping in Chrome and Firefox, and it will be fully supported in macOS High Sierra and iOS 11, with Microsoft Edge support following shortly thereafter. But what is WebAssembly and what does it mean for the web?

WebAssembly: A New Hope

Several years ago, I was working on a platform for photographers to schedule the publication of their best shots on different platforms. You may think of it as Buffer for photographers. To drive more traffic to the website, I was struggling with finding ways to attract potential customers.