Today we’re releasing Mobile Tethering, a new feature that brings Faraday to your mobile devices.

It’s a new way to access your local AI models from anywhere, without sacrificing privacy or performance. Simply leave the Faraday Desktop App running in the background on your Mac or PC at home, and you’ll be able to spin up LLMs, chat with custom AI characters, and more from the convenience of your mobile browser. Everything still runs on your own hardware, so it’s 100% free to use.

Tethering is completely opt-in, not enabled by default, and can be turned off anytime. The desktop app can always be used without an internet connection.

Overview

What is Faraday?

Faraday is a desktop app for creating custom AI chatbots (also known as characters) powered by LLMs that run locally on your Mac or PC. The installation process is dead-simple and requires no configuration; local chat “just works” out of the box. Faraday can run on CPU with only 8GB of RAM, and if you have a GPU, the app will automatically use it to speed things up.

What is Tethering?

Tethering extends all the capabilities of Faraday’s desktop experience to mobile, by allowing you to utilize your computer’s local resources remotely from a web browser. That means you can access all your existing models, characters, and chat conversations on the go, while still preserving the fundamental advantages of running Faraday on local hardware:

• Privacy - your chat data is generated and stored directly on your home computer
• Dead-simple to set up - run local AI chat with zero configuration
• 100% free to use - unlimited generations at zero cost

You can enable Tethering with one click, and we’ve built a new mobile-friendly UI that makes all of Faraday’s features easily accessible on smaller screens.

How to use it

1. First, make sure you are logged in to the same Faraday account in your desktop app and mobile browser.
   • If you don’t have the Faraday Desktop app, please download it.
   • If you don’t have an account yet, please sign up.
2. On Desktop, click on Settings > Tethering tab > set the Tethering toggle to “Enabled”.
3. In the Mobile Browser, click on Settings > set the Server toggle to “Tethering.”

For a step-by-step walkthrough with screenshots, you can also consult our docs page on Tethering.

Building Tethering

Tethering enables you to share Faraday’s desktop infrastructure between the local desktop app and the mobile browser. This section describes how we built this new functionality.

App Architecture

Here is what our desktop app and web infrastructure looked like before Tethering:

This should look familiar to most developers who have used tRPC. For those who haven’t, we’d recommend reading through the docs. We love tRPC at Faraday and are proud sponsors of the project. It enables us to develop features quickly, easily refactor code, and share logic across our desktop, web, and (soon to be released) mobile app.

A fully local tRPC pipeline (shown in the dotted blue box above) worked quite well for us when we first launched, but as soon as we wanted to give our users the ability to share and download characters online via our Character Hub, we needed to introduce some way of retrieving data from an online service and displaying it inside the desktop app.

After exploring a few different options, we introduced a Request Router to our desktop app. With some Typescript & tRPC hackery, we added the following logic:

• Have one TRPC client for both the desktop & web tRPC routes.
• If it’s a Character Hub or Cloud model request, forward the request to the web app tRPC server.
• Otherwise, forward the request to the desktop tRPC server.

This was a massive win, as we could reuse any React component on the web or desktop without sacrificing UX or DevEx (the only unshared code is a set of typed errors that can happen when calling remote resources, such as offline, auth, 500s, etc.). We have been running this archetype in production for months and have found it easy to reason about and debug.

Implementing Tethering

To support Tethering, we came up with an elegant solution that required only one new addition to our web infrastructure: a second Request Router. Here’s a diagram of the new architecture, with the new addition highlighted in green:

The Request Router in the web app has the following responsibilities:

• If an incoming request is intended for the Character Hub, always send it to the web tRPC server.
• If the user has enabled Tethering, send all other (non-Character Hub) requests to the user’s machine to be processed by the desktop application.

Some more logic is required that we won’t cover here, such as authentication and request shape validation (Zod is helpful). These are handled once globally, rather than inside the TRPC routes.

Overall, almost all of our tethering logic lives inside of the Request Router. This is a great pattern for us, as it allows us to be able to avoid repeated branching logic across the app (e.g. is_tethering booleans in every function/tRPC route).

Notes on security

• All messages between services are encrypted in transit. Messages between the Mobile Browser, the Request Router, and the tRPC Electron server are always sent over HTTPS.
• The Request Router signs all message requests to prevent response hijacking by malicious clients.
• Chat data is only stored on your local hard drive, ensuring complete privacy. We never log or store chat data for any reason.
• Unlike other solutions that offer remote access to local resources, such as port forwarding, our Tethering implementation does not require you to open any ports on your machine. This drastically reduces the surface area of possible security issues, makes HTTPS default, and enables Tethering in various network/firewall configurations.

If this interests you and you want to help solve other interesting problems at Faraday, we’re hiring a Senior Full-stack Engineer! See our job posting for more details.

To share your feedback, please join us on Discord, Reddit, or Twitter.