Francisco Sousa's blog

Leveling up your web apps with Zod and ChatGPT

Francisco Sousa — Wed, 20 Dec 2023 00:00:00 GMT

It's been pretty much a year since ChatGPT became available to the general public. Time goes pretty fast. Today, OpenAI's products are better and faster than ever, and the competition is pretty healthy with alternatives like Anthropic and Bard.

A lot of tech companies across the industry are either pivoting to AI or extending their products with AI features. This leaves us, the engineers of said companies, an entirely new way of thinking about the products we build. Let's see what we, JavaScript folks, can do to integrate these powerful tools with our existing web apps.

Pick an LLM provider

The first step is to pick an LLM provider. Most tech companies already have some sort of deal with providers like OpenAI. If your company isn't in this situation, you can suggest one. I'm biased towards OpenAI for their enterprise agreement and data protection policies.

Keep costs in mind. If you're with a big company with a substantial AI budget, go ahead. Otherwise, plan your budget carefully. LLM providers are expensive and offer no free tiers.

Companies often provide AI features only in paid plans or with strict daily/weekly/monthly limits, like Notion, to manage costs.

For this blog post, we'll proceed with the OpenAI API.

How to get JSON out of LLMs

LLMs take human language and out comes... "more language." That's the basic "black box" behavior we can assume from LLMs. This is not ideal if you are planning to integrate your existing APIs and systems with it, right? How do we even go about parsing whatever comes out of the LLM?

Easy, we tell the LLM to output in JSON. Yes, that's a thing. You can literally ask ChatGPT to respond in JSON. Even better, you can tell ChatGPT (and most LLMs) to answer you back according to a specific JSON schema specification that you yourself provide.

We can go up another level and build a JSON schema, asking the LLM to follow said schema. However, writing JSON schemas is not very pleasant, so we use something a little better, zod, a runtime validation library for JavaScript.

With zod, you can define schemas and validate your data against them. You can also infer Typescript types from zod schemas. This ensures type safety as well as runtime safety. You can then convert the zod schema to a JSON schema and ask the LLM to respond following the specified schema.

Luckily, there's already a great library that does this, zod-gpt. Here's how it works.

import z from "zod";
import { OpenAIChatApi } from "llm-api";
import { completion } from "zod-gpt";

const openai = new OpenAIChatApi(
  { apiKey: "YOUR-OPENAI-KEY" },
  { model: "gpt-3.5-turbo-16k" }
);

const schema = z.object({
    name: z.string().describe("The name of the person"),
    age: z.number().describe("The person's age")
});

const response = await completion(openai, "Imagine a random person", { schema });

// data will be typed as { name: string; age: number }
console.log(response.data);

Internally, zod-gpt will make sure the output from the LLM matches your provided zod schema, and it will even retry in case it fails. It's also aware of rate limits and will wait accordingly!

Some practical examples

Generating random people is not a very good use case, but one of the best applications for these LLMs is taking natural text and converting it into a structure.

For example, when I worked in healthcare tech, a popular challenge was taking doctor's notes and creating structured data out of them. Here's how you could do it:

import z from "zod";
import { OpenAIChatApi } from "llm-api";
import { completion } from "zod-gpt";

const openai = new OpenAIChatApi(
  { apiKey: "YOUR-OPENAI-KEY" },
  { model: "gpt-3.5-turbo-16k" }
);

const schema is z.object({
  name: z.string().describe("The patient's name"),
  age: z.number().describe("The patient's age"),
  weight: z.number().describe("The patient's weight in kilograms"),
  height: z.number().describe("The patient's height in centimeters"),
  bmi: z.number().describe("The patient's BMI"),
  bmiEnum: z
    .enum(["underweight", "normal", "overweight", "obese", "very_obese"])
    .describe("The patient's BMI in text format"),
  existingConditions: z
    .array(z.string())
    .describe("The patient's existing conditions"),
});

function parseDoctorsNotes(notes: string) {
  const prompt = `
    Please take these notes from a doctor
    and try to structure the data following the provided
    JSON schema info. Notes: ${notes}
  `;

  return completion(openai, prompt, { schema });
}

const response = await parseDoctorsNotes(
  "John Doe, 46 years. Had a cardiac arrest 2 years ago. 91kg, 184cm"
);

console.log(response.data);

/**
  Example output:

  {
    name: "John Doe",
    age: 46,
    weight: 91,
    height: 184,
    bmi: 26.86,
    bmiEnum: "overweight",
    existingConditions: ["cardiac arrest"]
  }
 */

This pattern is extremely useful for any boring, repetitive pattern your applications might have. Form filling is one of them and they can often be replaced with natural language parsing through LLMs.

Warning: This code is meant for servers!

The code above uses an API key from OpenAI. You should never ever store these keys on your client-side applications. Always run them on a server environment.

The example here would work for node, Deno, or bun. The only thing that would change is probably how you import the dependencies and how you access your environment variable.

Most popular frontend frameworks, like Next.js, Remix, or Svelte-Kit, have ways to run server code either through server-side rendering or API routes. Use those!

By running this on the server you can safely handle your API keys, and you can even implement rate limiting mechanisms so that your clients don't hammer your LLM APIs.

UX considerations

However, all of this technology needs to be used with a different UI/UX perspective in mind. AI is not 100% reliable. When you talk with any chat assistant they make sure to tell you that results might not be 100% correct and to take them with a grain of salt. But these usages of LLMs often don't pass this info to our users, so we have to do it ourselves.

In the real world, this doctor example should pre-fill an existing form component instead of just submitting this data somewhere. This way, doctors could double-check the info generated by the LLM before submitting.

And please make sure you are transparent with users about the use of AI and always make them opt-in or easy to bypass with a manual alternative!

Final thoughts

I hope this discovery of zod-gpt unlocks some ideas for you! It might be an obvious solution for some, but for me and others, thinking about LLMs this way was truly mind-blowing.

This wouldn't be possible without the work of David Zhang, who created both the llm-api and zod-gpt packages! Thank you.

Validating environment variables with zod

Francisco Sousa — Sat, 18 Mar 2023 00:00:00 GMT

This is a quick blog post to showcase something I've seen in some frontend starter repositories, namely create-t3-app.

Environment variables and .env files are the de-facto way of configuring JavaScript web applications. Most meta-frameworks, such as Next.js, Svelte Kit, and others, support them out of the box, allowing you to have one or multiple .env files for your projects.

Some frameworks even add TypeScript support, generating types based on the contents of your .env files, as svelte-kit does.

Taking it a step further

Even with TypeScript support, there is one thing that I think is essential for most web applications: runtime validation of used environment variables.

This is extremely useful when other developers check out your projects. It can be difficult to figure out exactly what environment variables are needed and what values they should have, unless you keep a detailed documentation of every environment variable (which you should do, even if you use the information in this blog post).

And as it’s usual for runtime validations, we can use zod to perform this task:

import z from "zod";

const envSchema = z.object({
  DATABASE_URL: z.string(),
  NODE_ENV: z.enum(["development", "test", "production"]).default("development"),
});

export const ENV = envSchema.parse(process.env);

In the above code, we define a zod schema that requires the presence of a DATABASE_URL and NODE_ENV. The value of NODE_ENV defaults to development.

Next, we export an ENV variable with the result of the parse function from our zod schema. The parse function validates the contents of process.env and returns an object that matches the schema. This means we filter out any unnecessary information from process.env. Keep this in mind.

We can use it as such:

import { ENV } from "./env.ts";

// This will use be fully typed if using Typescript! Will also work for Javascript.
console.log(ENV.NODE_ENV);

When using TypeScript, all uses of ENV will be strongly typed. Additionally, if the schema has validation errors, the application itself will crash if DATABASE_URL is missing, for example.

Handling non-string values

If you need to read non-string values such as numbers or booleans from your environment, you can rely on the transform function.

import z from "zod";

const envSchema = z.object({
    DATABASE_URL: z.string(),
    NODE_ENV: z.enum(["development", "test", "production"]).default("development"),
    SOME_NUMBER: z.string().regex(/^\\d+$/).transform(Number),
    SOME_BOOLEAN: z.enum(["true", "false"]).transform((v) => v === "true"),
});

export const ENV = envSchema.parse(process.env);

The strategy here is to first validate the string value, and then use transform to convert it to the final value. For numbers, a simple regex can be used, and for booleans, an enum can be used. When using ENV, SOME_NUMBER and SOME_BOOLEAN will have a type of number and boolean, respectively.

Handling a bad schema

If our environment doesn't match our defined schema, our application could crash during runtime. Some frameworks and libraries might actually crash when running the app server itself, which is good, but others might crash during runtime. To prevent this, it is a good practice to validate the schema before running our application.

First, we need to find a way to extract issues from our schema, as it will be useful when running the validation script.

import z from "zod";

const envSchema = z.object({
  DATABASE_URL: z.string(),
  NODE_ENV: z.enum(["development", "test", "production"]).default("development"),
});

export const ENV = envSchema.parse(process.env);

export const getEnvIssues = (): z.ZodIssue[] | void => {
  const result = serverEnvSchema.safeParse(process.env);
  if (!result.success) return result.error.issues;
};

Next, we write a validation script that we can run before launching our web app. You can even incorporate this step into your CI pipelines, to detect environment errors before actually deploying your application. I personally add this script to my build step on Vercel or Netlify (or wherever I'm deploying).

Make sure to install zod-error as well. It helps to prettify zod errors for you.

import { generateErrorMessage } from "zod-error";
import { getEnvIssues } from "./env.ts";

const issues = getEnvIssues();

if (issues) {
  console.error("Invalid environment variables, check the errors below!");
  console.error(
    generateErrorMessage(issues, {
      delimiter: { error: "\\n" },
    })
  );
  process.exit(-1);
}

console.log("The environment variables are valid!");

If you are working in a TypeScript environment, you can run the script above using ts-node.

I can’t use `process.env`!

Users of tools like deno, Cloudflare Workers, vite, or bun cannot use process.env, which is specific to Node.js (although it also works on Webpack!).

Instead, here are some alternatives:

deno - use Deno.env.toObject()
Cloudflare - while not tested, you can try to access environment variables as global variables via globalThis
vite - use import.meta.env
bun - use Bun.env

In summary, consult the documentation for your runtime or meta-framework to learn how to access the objects containing all environment variables.

Hope this information is helpful! See you soon.

Using zustand with Svelte

Francisco Sousa — Sun, 09 Oct 2022 00:00:00 GMT

zustand is an excellent library primarily used in the React community. It's my go-to choice for any shared state management in React due to its ease of use, and for being a solution to most shared state issues like zombie childs, react concurrency, and context loss between mixed renderers.

For these same reasons, I also started using it in some Svelte projects. I like how short and simple most zustand stores look. Even though Svelte built-in store mechanisms are excellent, we can go up a level by combining the vanilla solution and zustand. We can have the best of both worlds.

What's a Svelte store?

According to Svelte's docs, which I recommend reading, a Svelte store can be any object with a subscribe method. However, it's recommended that you use Svelte's tools to create writable, readable, derived, or custom stores. Personally, I most often default to custom stores.

An example of one such store:

import { writable } from 'svelte/store';

function createCount() {
  const { subscribe, set, update } = writable<number>(0);

  return {
    subscribe,
    increment: () => update((n) => n + 1),
    reset: () => set(0)
  };
}

const countStore = createCount();

export default countStore;

To use the store in Svelte, you can do this:

<script>
  import counterStore from '../lib/counter.store';
</script>

<h1>Welcome!</h1>
<p>{$counterStore}</p>
<button on:click={() => counterStore.increment()}>Click</button>

The good thing about native Svelte stores is the auto-subscription feature. Within a Svelte component, you can automatically use the dollar sign to subscribe to a Svelte store. In the markup we use $counterStore. This allows us to read the store value immediately and react to its updates. That markup will always contain the latest $counterStore value.

In this example, our value is just a number, but the store's value can be anything, an object even!

Zustand

These Svelte stores are more than good enough! I've shipped production apps that used this to great effect. However, zustand has some advantages that can be very useful! You can connect zustand stores to Redux devtools using their devtools middleware. There is also a middleware that you can use to persist any store with any storage provider. You can also easily craft your middleware, for example, to connect your store to the URL hash to save filters on a complex table component.

Let's turn our previous counter store into a zustand store:

import create from 'zustand/vanilla';
import zustandToSvelte from './zustandToSvelte';

export interface CounterState {
  value: number;
  increment: () => void;
}

const counterStore = create<CounterState>((set) => ({
  value: 0,
  increment: () => set((state) => ({ value: state.value + 1 }))
}));

export default counterStore;

zustand can infer types from your store (if you use their combine utility), but it's recommended to write them yourself. You must also type the actions themselves, as values and actions are all part of the store.

Still, this is not ready to be used by Svelte. We need to turn this into a native store first! zustand stores have a subscribe method, but it's not natively compatible with Svelte. Let's create a wrapper that turns a zustand store into a compatible Svelte store.

import { readable } from 'svelte/store';
import type { StoreApi } from 'zustand';

export default function zustandToSvelte<StateType>(zustandStore: StoreApi<StateType>) {
  const svelteStore = readable(zustandStore.getState(), (set) => {
    zustandStore.subscribe((value) => set(value));
  });

  return {
    ...zustandStore,
    subscribe: svelteStore.subscribe
  };
}

Here we create a readable Svelte store from our zustand store. readable takes two arguments: the initial state and a function that can set the state of the readable. In this function, we can subscribe to our store and set the state of the native Svelte readable store.

Then, we return the original zustandStore with the overridden subscribe method from our readable Svelte store. zustand subscribe doesn't work by default on Svelte components. That's why we need to go through all of these steps.

All it's left, is to go back to our zustand store example and use this utility:

import create from 'zustand/vanilla';
import zustandToSvelte from './zustandToSvelte';

export interface CounterState {
  value: number;
  increment: () => void;
}

// Wrap this with `zustandToSvelte`
const counterStore = zustandToSvelte(create<CounterState>((set) => ({
  value: 0,
  increment: () => set((state) => ({ value: state.value + 1 }))
})));

export default counterStore;

We can then use our new store like so:

<script>
  import counterStore from '../lib/counter.store';
</script>

<h1>Welcome!</h1>
<p>{$counterStore.value}</p>
<button on:click={() => $counterStore.increment()}>Click</button>

Unlike a native Svelte store, the whole store object must be subscribed to. In our native example, you can call counterStore.increment() without the dollar sign. With these new zustand stores, you'll always have to use the dollar sign: $counterStore.increment().

Outside of Svelte components, you can use its methods as usual:

import counterStore from '../lib/counter.store';

// You can peek into the state or even call some actions
counterStore.getState().increment()

console.log(counterStore.getState().value)

// You can set the state
counterStore.setState({ value: 0 })

// You can also subscribe to it!
counterStore.subscribe((state) => console.log(state));

Final thoughts

To be honest, Svelte custom stores are almost as powerful as zustand. I'm just more familiar with the latter, and their middleware tools are also a great nice to have. However, this pattern can be applied to many state management technologies. Whether you have some redux store or xstate machine that you want to use within Svelte components, you can reach for svelte/readable to try and make it work!

Stay safe!

Typing remix loaders with confidence

Francisco Sousa — Sun, 19 Jun 2022 00:00:00 GMT

Well, hello there, it's time for a lightning-quick Remix tip. Let's see how we can genuinely write typesafe Remix routes by sharing types between loaders and components for full-stack typing!

Remix, what?

For the readers unfamiliar with Remix, it's a React framework created by the react-router team. It uses react-router to make a server-rendering full-stack framework with React support. It's the other kind of Next.js (sorry).

Loaders, what are they?

Remix is a server-side rendering framework, and as such, you can load data directly to your components while they are being rendered on the server.

export function loader() {
  return "Hello world!";
}

export default function SomeRemixPage() {
  const data = useLoaderData();
  return <p>{ data }</p>;
}

You can only define the loader function on the Remix route file, but you can then call the useLoaderData hook on every component used inside that route. This is very useful for better SEO and spares you from adding loading states to your app, as the page comes pre-rendered from the server.

Let's add types the regular way

You can quickly type useLoaderData using type variables. Its type signature is useLoaderData<T>: T, so if you do useLoaderData<string>, you just typed your loader!

export function loader(): string {
  return "Hello world!";
}

export default function SomeRemixPage() {
  const data = useLoaderData<string>();
  return <p>{ data }</p>;
}

However, this has a couple of issues. Typing the generic T on useLoaderData is the same thing as doing this:

const data = useLoaderData<string>();
const data = useLoaderData() as string;

If you do not type useLoaderData, its default type is any, so you can just cast that to whatever you want. This means that the following scenario won't report type errors and would just crash during runtime.

export function loader(): string {
  return "Hello world!";
}

export default function SomeRemixPage() {
  const { data } = useLoaderData<{data: string}>();
  return <p>{ data }</p>;
}

In the above scenario, this will crash, even though the types are all valid. We want the loader data to be of type { data: string }, but as there is no direct-type connection between the loader and the hook, some bugs might leak into runtime if you type all of your loader and useLoaderData like this.

Inferring the types from the `loader`

The solution is to infer the types from the loader automatically. The first step is to never use the LoaderFunction type.

import { json } from '@remix-run/node'; // or "@remix-run/cloudflare"
import type { LoaderFunction } from '@remix-run/node'; // or "@remix-run/cloudflare"

export const loader: LoaderFunction = async () => {
	return json({ ok: true });
};

As of Remix version 1.5.1 the LoaderFunction return type is Promise<Response> | Response | Promise<AppData> | AppData which means we cannot reliably use the solution I will propose. AppData is an internal Remix type that is the same as any, which doesn't do much for type safety.

The second step is to never type the return value of the loader function. We are going to do that automatically from now on. So if you have any export function loader(): SomeType, make sure you remove the SomeType from there.

Then we can start inferring the type of our loader automatically!

type LoaderType = Awaited<ReturnType<typeof loader>>;

This essentially infers the type of the loader function.

Awaited extracts the type of a promise because loader can be async
ReturnType is pretty straightforward and returns the type returned by typeof loader

Revisiting our previous example, it would become this:

export function loader(): string {
  return "Hello world!";
}

type LoaderType = Awaited<ReturnType<typeof loader>>;

export default function SomeRemixPage() {
  const { data } = useLoaderData<LoaderType>();
  return <p>{ data }</p>;
}

Typescript would then complain that there is no property data on type string! We can fix that by correctly refactoring our loader.

export function loader() {
  return { data: "Hello world!" };
}

type LoaderType = Awaited<ReturnType<typeof loader>>;

export default function SomeRemixPage() {
  const { data } = useLoaderData<LoaderType>();
  return <p>{ data }</p>;
}

If you want to type the arguments of loader you can import the following from Remix internals:

import type { DataFunctionArgs } from "@remix-run/server-runtime";

export function loader(({ request }: DataFunctionArgs)) {
  // do stuff
}

This will keep the return type untouched so you can automatically infer it.

This solution is a great help because it also takes care of conditionals! Imagine that this page is only for authenticated users:

export function loader({ request }: DataFunctionArgs) {
  if (!extractUserFromRequest(request)) return new Response(null, { status: 401 });
  return { data: "Hello world!" };
}

type LoaderType = Awaited<ReturnType<typeof loader>>;

export default function SomeRemixPage() {
  const { data } = useLoaderData<LoaderType>();
  return <p>{ data }</p>;
}

Here, Typescript would complain that there is no data on Response when using the useLoaderData hook. This would avoid a regression here. You could quickly fix this by using throw when checking for the user session instead of return. Remember that you can throw inside a loader function to immediately stop rendering! It would also keep Typescript silent because the function only returns { data: string }.

Final notes

You can also export the types from the inferred loader functions to use wherever you want. This lets us ensure everything is nice and tidy and keep runtime errors to a minimum.

Hope this helped, let me know if you have any questions! If you also have a better solution than this one, please let me know!

Stay safe out there!

Locale detection with Remix

Francisco Sousa — Tue, 18 Jan 2022 00:00:00 GMT

Lately, I've been fooling around with remix.run and server rendering apps instead of client-side only stuff. However, whenever I use things like toLocaleString from the Date API, we get shifting date formats on the frontend, and Text content did not match errors from React. This warning often happens in either Remix or Next.js, or any SSR (server-side rendering) React-based solution. Let's see why it happens and how we can fix it.

Why does this happen when using SSR?

This warning is common for devs getting started with SSR and using Javascript APIs that access the locale, like toLocaleString on the Date API. Often, the locale of the Node.js runtime on the server will not match your browser's locale, so when your app server renders an HTML page, it will use the default locale of that particular server, usually en-us. When your browser re-hydrates the page, it will use your default locale. If it doesn't match the en-us locale, your toLocaleString call will most likely be different from the server, and you will get that warning, and you may sometimes even notice the date flashing back from one format to another.

How can we fix it then?

Well, the quick approach is to hardcode en-us everywhere. Most locale-based APIs accept a locale or an array of locales as an argument. If your website or app only is available in one language, it might even be a good idea just to show it with a single locale to avoid UI confusion. (...).toLocaleString("en-us") would fix it by using that locale always on the server and the client.

However, most SSR frameworks can access the headers of an HTTP request, which means you can access the Accept-Language HTTP header! You can check out in the MDN docs what the header is and its format. Essentially it tells servers the list of languages the user accepts.

Why a list, though? Because most browsers allow you to set a list of preferred languages ordered by your preference. Take a look at mine (you can access the page via chrome://settings/languages).

My solution is simple. We get the header value from the request, then pass it into a Provider. Then we will create a hook to get the locale value whenever we need it. We will only extract the user's first language (the favorite one).

We will use the accept-language-parser package to parse the header value. Let's install it

# or npm if you are into that
yarn add accept-language-parser

# if you are using typescript, this might be useful
yarn add --dev @types/accept-language-parser

Now, let's create our provider/context/hook combo. I'll put it under app/hooks/useLocale.tsx. Our default export is the hook, but we also export the provider to use it on the root file of remix.

import { createContext, ReactNode, useContext } from "react";

const LocaleContext = createContext<string>("");

export function LocaleProvider({
  children,
  locale,
}: {
  children: ReactNode;
  locale: string;
}) {
  return (
    <LocaleContext.Provider value={locale}>{children}</LocaleContext.Provider>
  );
}

export default function useLocale() {
  return useContext(LocaleContext);
}

Then create a new loader in your root file. Mine is in the app/root.tsx file. If you already have a loader, augment it with this one.

import type { LoaderFunction } from "remix";
import acceptLanguage from "accept-language-parser";

export const loader: LoaderFunction = async ({ request }) => {
  const languages = acceptLanguage.parse(
    request.headers.get("Accept-Language") as string
  );

  // If somehow the header is empty, return a default locale
  if (languages?.length < 1) return "en-us";

  // If there is no region for this locale, just return the code
  if (!languages[0].region) return languages[0].code;

  return `${languages[0].code}-${languages[0].region.toLowerCase()}`;
};

Then, wrap it with our provider in your default export of the root file.

import { useLoaderData } from "remix";
import { LocaleProvider } from "~/hooks/useLocale";

export default function App() {
  const locale = useLoaderData();

  return (
    <LocaleProvider locale={locale}>
      {/* Whatever your already had here */}
    </LocaleProvider>
  );
}

And that's it. Any component on your remix routes can now use useLocale to get the current locale! An example:

import useLocale from "~/hooks/useLocale";

export default function HomePage() {
  const locale = useLocale();

  return (
    <div>
      {1_000_000.toLocaleString(locale)}
    </div>
  );
}

For Next.js the solution is very similar! Just play around with this concept using getInitialProps on the _app.js component.

I hope this solution is of use to you! You can see it in action on my remix starter repo with prisma support and auth via cookies.

Stay safe!

Authenticating things with cookies on Next.js

Francisco Sousa — Sat, 31 Jul 2021 00:00:00 GMT

Originally posted on Finiam's website.

Please click here to skip all of the backstory behind this blog post and jump straight into the code!

A while back I made a blog post on how to manage user authentication on React apps. It's a task I've made time and time again and decided to share this with you folks. That blog post is aimed at single-page apps obviously, but nowadays we are seeing more and more (again) server-rendered apps, especially with Next.js.

When server-rendering pages, authentication works a little differently. While you might use the context API or any other kind of mechanism to authenticate users on your app on the client-side, we should do that on the server so that users receive their content in the context of their authentication status. Let's see how.

Next.js

Next.js in short is a React app framework that server renders components. If you use React you probably heard about Next.js. If you didn't, please take some time to read about it on their website, it's great and it's pretty much the only way we write React apps at Finiam.

Next.js allows us to server render components and play with the usual req and res objects on an HTTP request lifecycle, something that you can't really do with a traditional React app SPA. This means we can interact with the cookies involved in our app.

However, in Next.js there is no need to handle that on the client side. You can directly authenticate users in getServerSideProps and immediately render the page based on the authentication status, without having to deal with that on the frontend.

It also means that you get access to a node.js environment, meaning that you can even use a database directly instead of having to develop a remote API yourself. It's full-stack React development!

A quick taste of how that can look like.

By the way, we pretty much just write Typescript nowadays. You should really be writing Typescript.

import React from 'react';
import { GetServerSidePropsContext } from 'next';
import Head from 'next/head';
import Login from 'root/components/Login';
import { userFromRequest } from 'root/web/tokens';

interface User {
	email: string;
	name: string;
}

interface Props {
	user?: User;
}

export default function Home({ user }: Props) {
	if (!user) return <Login />;

	return <div>Hello {user.name}</div>;
}

export async function getServerSideProps(context: GetServerSidePropsContext) {
	const user = await userFromRequest(context.req);

	if (!user) return { props: {} };

	return {
		props: { user }
	};
}

Awesome right? In this example, the magic is on the userFromRequest method, which extracts the user by reading the cookie that was previously set during login.

Let's dive into the technicalities.

Architecture

The small project we are going to make consists of a very simple but relatively secure sign-up and login flow, and a user-owned resource, in our case, just some notes that users can write for themselves. Users can authenticate themselves and write notes that can only be accessed by them.

We are going to store users and their hashed passwords on our database. When users sign up or login successfully we place a JWT token that contains the email of said user, and we sign this JWT with a secret key (that should be random and crypto secure). This token is then stored on an HttpOnly and Secure cookie, so that it's only readable on the server-side and no client-side Javascript can access it. The cookie will be valid for 1 day, but this can be configured as you prefer.

Whenever we want to identify the user behind a network request, we check the cookies of said request, take our JWT token, and then check if it was signed with our secret key, so bad actors can't forge JWT tokens. If it was signed with our key we can assume that it's a valid token. Then we fetch the user info from the database and we can use it freely to build pages on the server-side.

The users' passwords are all hashed using argon2, the strongest and safest hashing algorithm out there, that won the PHC (Password Hashing Competition) very recently.

Disclaimer

The code you are about to see, especially the backend part of it, lacks a lot of the required functionality to make a truly robust and secure authentication flow. It doesn't handle brute force attacks, doesn't rate-limit sign-ups, has no geolocation protection, has no session management, meaning that we can't log out users remotely or anything like that, and has no CSRF protection. We work in the finance world where most of these things are mandatory. We recommend using auth0 which has most of these features or at least setting up Cloudflare to rate the limit on your endpoints.

Our password hashing algorithm is safe, the strongest in the industry. The JWT tokens are being generated correctly, but it's recommended to rotate the signing key daily or weekly. It's enough to secure most web apps, but if you are handling sensitive data, you should take more steps to secure it.

We usually write most of our complex backend apps with Phoenix or Rails. In these frameworks, you have much more complete libraries that handle most of the steps we are going to make by hand!

Still, you also have NextAuth.js which is a pretty nice project that handles integration with 3rd parties, implement CSRF and session management. But still lacks brute force and geolocation protection.

Consider this a learning project, that can go into production, but don't implement, let's say, a banking system using it without added protection!

Let's get coding

To get started just bootstrap a Next.js example project:

yarn create next-app --typescript

All of the code I'm going to show is a simplified version of what's present on our next-prisma-starter template at the time of writing this blog post.

User management

Let's create a Prisma schema for our app: prisma/schema.prisma

generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgres"
  url      = env("DATABASE_URL")
}

model User {
  id         String        @id @default(uuid())
  email      String        @unique
  name       String
  password   String
  createdAt  DateTime      @default(now())
  updatedAt  DateTime      @updatedAt
  Note       Note[]
}

This schema will create a database where we can store users and each user has many notes.

You can then use the Prisma migration tool to apply these changes to your database.

prisma migrate dev --name init
prisma migrate deploy --preview-feature && yarn db:generate

Now, let's create our lib layer. On full-stack Next.js projects, I like to put all of the web-related code on src and then all of the backend business logic on a lib directory. Both, at the root of the project.

First, the users module.

import { User } from '@prisma/client';
import prisma from 'lib/prisma';
import { encryptPassword } from 'lib/auth/passwordUtils';
import pick from 'lodash/pick';

export interface UserParams {
	email: string;
	name: string;
	password: string;
}

// Given some params, create a user on the database,
// storing the encrypted password.
export async function createUser(params: UserParams): Promise<User> {
	const filteredParams = pick(params, ['email', 'name', 'password']);
	const password = await encryptPassword(filteredParams.password);
	const user = await prisma.user.create({
		data: { ...filteredParams, password }
	});

	// Make sure all our lib methods obfuscate the password
	user.password = '';

	return user;
}

Then, the auth module.

import { User } from '@prisma/client';
import prisma from 'lib/prisma';
import { verifyPassword } from './passwordUtils';

export interface LoginParams {
	email: string;
	password: string;
}

// Given some login params (email and password)
// return the user if the password is valid
// or null if it's not.
export async function login(params: LoginParams): Promise<User> {
	const user = await prisma.user.findUnique({ where: { email: params.email } });

	if (!user) return null;

	if (await verifyPassword(user.password, params.password)) {
		// Make sure all our lib methods obfuscate the password
		user.password = '';

		return user;
	}

	return null;
}

Now the final missing piece, the hashing algorithm.

import argon2 from 'argon2';

export async function encryptPassword(password: string): Promise<string> {
	return argon2.hash(password);
}

export async function verifyPassword(hash: string, password: string): Promise<boolean> {
	return argon2.verify(hash, password);
}

Now we can create and login users easily.

Cookies and JWT

You probably heard about JWT also known as JSON web tokens. They are used by most web services for authentication, using JWT to encode some of the user info and use it as a token that identifies the user. You should never store secret info on JWTs as they are meant to be readable by anyone. It's the signing mechanism of a JWT however that guarantees the identity and validity of a given token.

When generating a JWT we sign it with a very secret key, and then we can verify that any subsequent JWT passed to us was correctly generated with said very secret key. If any attacker forges a JWT, that won't work as they are not generated with said key.

Like most web services, we generate a JWT with the user email (or any other unique identifier of the user) in it on a successful login or sign-up and then use server-side secure cookies to store it. This way, it's impossible for malicious Javascript to get the cookie. It's also pretty easy to directly, on the server-side, render the user's authenticated state.

The code now. We are going to save this file under src/web/tokens.ts. This is related to web logic and not exactly business-side logic. Our module exports 3 functions: authenticateUser, clearUser and userFromRequest

import { User } from "@prisma/client";
import jwt from "jsonwebtoken";
import { serialize } from "cookie";
import { NextApiResponse } from "next";
import prisma from "lib/prisma";
import { NextApiRequestCookies } from "next/dist/next-server/server/api-utils";
import { IncomingMessage } from "http";

// You should really not use the fallback and perhaps
// throw an error if this value is not set!
const JWT_TOKEN_KEY = process.env.JWT_TOKEN_KEY || "super duper secret key";
const cookieOptions = {
  httpOnly: true,
  maxAge: 2592000,
  path: "/",
  sameSite: "Strict",
  secure: process.env.NODE_ENV === "production",
};

function setCookie(
  res: any,
  name: string,
  value: string,
  options: Record<string, unknown> = {}
): void {
  const stringValue =
    typeof value === "object" ? `j:${JSON.stringify(value)}` : String(value);

  res.setHeader("Set-Cookie", serialize(name, String(stringValue, options));
}

// This sets the cookie on a NextApiResponse so we can authenticate
// users on API routes.
export function authenticateUser(res: NextApiResponse, user: User): void {
  if (!user) return;

  const token = jwt.sign({ email: user.email }, JWT_TOKEN_KEY, {
    expiresIn: "1d",
  });

  setCookie(res, "auth", token, cookieOptions);
}

// This removes the auth cookie, effectively logging out
// the user.
export function clearUser(res: NextApiResponse): void {
  setCookie(res, "auth", "0", {
    ...cookieOptions,
    path: "/",
    maxAge: 1,
  });
}

// This gives back the user behind a given request
// either on API routes or getServerSideProps
export async function userFromRequest(
  req: IncomingMessage & { cookies: NextApiRequestCookies }
): Promise<User | undefined> {
  const { auth: token } = req.cookies;

  if (!token) return undefined;

  try {
    const data = jwt.verify(token, JWT_TOKEN_KEY);

    if (!data) return undefined;

    const user = await prisma.user.findUnique({
      where: { email: (data as any).email },
    });

    if (user) user.password = "";

    return user;
  } catch (error) {
    return undefined;
  }
}

Now, we are able to create our API routes

API routes

We are using next-connect to implement our API routes. They have a cleaner API and allow us to easily use middlewares and the like.

Let's quickly create a default next-connect handler first. You can add default middlewares to this handler so we can re-use those on all of our API routes. In this, we can even define custom error handling behavior.

import { NextApiResponse } from 'next';
import nextConnect from 'next-connect';

export default function defaultHandler<ReqType, ResType>() {
	return nextConnect<ReqType, ResType>({
		attachParams: true,
		onError: (err, req, res) => {
			console.error(err);

			(res as unknown as NextApiResponse).status(500).json({ error: 'Internal Server Error' });
		}
	});
}

Now, for our API we are going to need two modules and a total of three endpoints.

/sessions
- POST - logs in a user with a email/password combo
- DELETE - logs out the user
/users
- POST - creates users

import { NextApiRequest, NextApiResponse } from 'next';
import { login } from 'lib/auth';
import { authenticateUser, clearUser } from 'root/web/tokens';
import defaultHandler from './_defaultHandler';

const handler = defaultHandler<NextApiRequest, NextApiResponse>()
	.post(async (req, res) => {
		const user = await login(req.body);

		if (user) {
			authenticateUser(res, user);
			res.json(user);
		} else {
			res.status(404).send('');
		}
	})
	.delete((_req, res) => {
		clearUser(res);

		res.send('');
	});

export default handler;

import { NextApiRequest, NextApiResponse } from 'next';
import { createUser } from 'lib/users';
import { authenticateUser } from 'src/web/tokens';
import defaultHandler from './_defaultHandler';

const handler = defaultHandler<NextApiRequest, NextApiResponse>().post(async (req, res) => {
	const user = await createUser(req.body);

	authenticateUser(res, user);
	res.json(user);
});

export default handler;

Now to go full circle, the frontend.

Frontend

On the frontend we need 4 dependencies, redaxios, react-hook-form , react-query and superjson. Feel free to go 100% vanilla on this, but we often end up installing this when projects become more complicated anyway.

Let's just make 2 pages, a sign-up page, and a homepage. The home page either shows the current user email or the login form.

Setup react-query first.

import React from 'react';
import { QueryClient, QueryClientProvider } from 'react-query';

const queryClient = new QueryClient();

export default function App({ Component, pageProps }) {
	return (
		<QueryClientProvider client={queryClient}>
			<Component {...pageProps} />
		</QueryClientProvider>
	);
}

Then, let's just implement a very nice utility that allows us to refresh server-side data from Next.js without a full page reload.

import { useRouter } from 'next/router';

export default function useServerRefresher(): () => void {
	const router = useRouter();

	return () => router.replace(router.asPath);
}

We can use this hook to refresh the data from getServerSideProps without a full page reload! This is very handy to update stale data.

Then, our sign-up page. We use this hook, after the successful createUser mutation, the server refresher gets called and then we re-run the code on getServerSideProps again, which redirects us to the homepage.

import React from 'react';
import Head from 'next/head';
import { useForm } from 'react-hook-form';
import Link from 'next/link';
import { GetServerSidePropsContext } from 'next';
import { useMutation } from 'react-query';
import useServerRefresher from 'src/hooks/useServerRefresher';
import { userFromRequest } from 'src/web/tokens';

export default function SignUp() {
	const {
		handleSubmit,
		register,
		formState: { errors }
	} = useForm();
	const {
		isLoading,
		isError,
		mutate: createUserMutation
	} = useMutation((params) => redaxios.post('/users', params), {
		onSuccess: useServerRefresher()
	});

	const handleCreateUser = (params) => createUserMutation(params);

	return (
		<main>
			<form
				className="h-screen center flex flex-col items-center justify-center"
				onSubmit={handleSubmit(handleCreateUser)}
			>
				<Head>
					<title>Sign Up</title>
				</Head>

				<div className="space-y-8">
					<h1 className="self-start text-xl">Sign up</h1>

					<label className="flex flex-col" htmlFor="email">
						Email
						<input id="email" type="email" {...register('email', { required: true })} />
					</label>

					<label className="flex flex-col" htmlFor="name">
						Name
						<input id="name" type="text" {...register('name', { required: true })} />
					</label>

					<label className="flex flex-col" htmlFor="password">
						Password
						<input id="password" type="password" {...register('password', { required: true })} />
					</label>

					<button
						className="u-button"
						type="submit"
						disabled={Object.keys(errors).length > 0 || isLoading}
					>
						Sign Up
					</button>

					{isError && <p>User exists</p>}

					<Link href="/">
						<a className="block underline" href="/">
							Login
						</a>
					</Link>
				</div>
			</form>
		</main>
	);
}

export async function getServerSideProps(context: GetServerSidePropsContext) {
	const user = await userFromRequest(context.req);

	if (user) {
		return {
			redirect: {
				destination: '/',
				permanent: false
			}
		};
	}

	return {
		props: {}
	};
}

And then, our homepage. In this case, we are not doing redirects. When having protected pages it's good to have the login logic on the component itself so users are still on the correct URL after logging in.

import React from 'react';
import { User } from '@prisma/client';
import { GetServerSidePropsContext } from 'next';
import Login from 'src/components/Login';
import { userFromRequest } from 'src/web/tokens';

interface Props {
	user?: User;
}

export default function Home({ user }: Props) {
	if (!user) return <Login />;

	const handleLogout = () => redaxios.delete('/sessions');

	return (
		<main className="max-w-4xl mx-auto py-20 space-y-8">
			Hello {user.name}!
			<button type="button" onClick={handleLogout}>
				Logout
			</button>
		</main>
	);
}

export async function getServerSideProps(context: GetServerSidePropsContext) {
	const user = await userFromRequest(context.req);

	if (!user) return { props: {} };

	// Always use superjson as Next.js
	// can't serialize prisma objects by default
	return {
		props: superjson.serialize({
			user
		}).json
	};
}

Don't forget the login component.

import React from 'react';
import Head from 'next/head';
import { useForm } from 'react-hook-form';
import { useMutation } from 'react-query';
import Link from 'next/link';
import useServerRefresher from 'src/hooks/useServerRefresher';

export default function Login() {
	const {
		handleSubmit,
		register,
		formState: { errors }
	} = useForm();
	const {
		isLoading,
		isError,
		mutate: loginMutation
	} = useMutation((params) => redaxios.post('/sessions', params), {
		onSuccess: useServerRefresher()
	});

	const onSubmit = async (params) => loginMutation(params);

	return (
		<form
			className="h-screen u-center flex flex-col items-center space-y-8"
			onSubmit={handleSubmit(onSubmit)}
		>
			<Head>
				<title>Login</title>
			</Head>

			<div className="space-y-8">
				<h1 className="self-start text-xl">Login</h1>

				<label className="flex flex-col" htmlFor="email">
					Email
					<input type="text" {...register('email', { required: true })} />
				</label>

				<label className="flex flex-col" htmlFor="password">
					Password
					<input type="password" {...register('password', { required: true })} />
				</label>

				<button
					className="u-button"
					type="submit"
					disabled={Object.keys(errors).length > 0 || isLoading}
				>
					Login
				</button>

				{isError && <p>User password combination not found</p>}

				<Link href="/signup">
					<a className="block underline" href="/signup">
						Sign up
					</a>
				</Link>
			</div>
		</form>
	);
}

Closing notes

This is a great way to get started on full-stack projects on Next.js, very useful for small to medium-sized projects. You get full SSR so users never get to see any loading spinner while getting authentication details, as you get on most SPAs and you get to iterate on features easily without having to maintain separate projects and technologies for backend and frontend.

You don't even need to use Prisma or do it all full-stack. You can have all of your business logic on a remote API and just use some Next.js API routes to manage authentication cookies and then fetch all your data on getServerSideProps. Really, the sky is the limit here.

If you really enjoyed the full-stack part of this, I recommend you looking into either Blitz.js or Redwood.js, they do most of the heavy lifting for you and have built-in features that are essential for app development. We only use our starter template for simpler stuff so we can enjoy the greater flexibility.

Predictable React authentication with the Context API

Francisco Sousa — Mon, 08 Mar 2021 00:00:00 GMT

Originally posted on Finiam's website.

Despite there being many React and authentication tutorials out there, I feel like I showcased this to too many people I mentored over the past few months, so, this is a good time to share it with a greater audience. Authentication is an app-wide concern, and therefore a global state issue. For many, keeping global state in React is a tricky task, however, React has had a nice way of handling "sort of" global state, with the context API and hooks. Let's see how.

Praise be the Context

useContext is our best bet today. I use it a ton for both complex, app-wide state management, or even on smaller multi-component APIs, like making a re-usable dropdown component (it's how the reach-ui folks do it). Kent’s blog post is a great way to learn a bit more about the context API and how to use it effectively if you have never heard of it before.

So, in order to manage authentication, we will use React’s context API to make it available for every component on the app, so you can easily implement classic login/logout/sign-up logic on your projects.

Some disclaimers

I am going to assume that you have some sort of backend already set up. The examples I am going to show you are present on our phoenix starter template. You can replace the API calls here with anything you have available. All of the code on this tutorial is there.

Also, this is probably not ideal for 3rd party OAuth providers. To integrate with providers like Auth0, Google, Facebook, and others, you should use their own SDKs instead of using the patterns I am going to show you. It’s just easier and their tools usually handle all of this.

On our work, at Finiam, we usually roll out our own authentication APIs together with the backend or we use whatever the client is using, which rarely includes OAuth providers.

Time to code

So, for our authentication management component, we have some basic requirements:

Allow login info to be submitted
Allow sign-up info to be submitted
Allow a user to logout
Check if a current user is logged in upon loading the app.

The plan is to provide these operations for the entire app using React’s context API and make them available with a simple useAuth hook, that allows us to read and manipulate the authentication.

Now the first step is to communicate with your authentication backend. We are going to make simple HTTP calls with redaxios. We just communicate with a few endpoints that manipulate server-side cookies to manage auth. There is no need to send authorization headers or manage tokens because all of the authentication is handled on the server-side and the browser just picks it up. We just make the HTTP calls and the server handles everything!

If your backend handles with something like JWT bearer tokens, you can use localStorage for that. You just need to modify your HTTP client to use the returned token on all of the following requests. You can also store it on local storage so users should not login every time. Be advised, that for web applications, server-side cookie authentication still offers the best security! Check this blog post for an accurate explanation about that. Avoid using localStorage.

The code to interact with the sessions API, which handles login and logout.

import redaxios from 'redaxios';

export async function login(params: { email: string; password: string }): Promise<User> {
	const response = await redaxios.post('/api/sessions', { session: params });

	return response.data.data;
}

export async function logout() {
	const response = await redaxios.delete('/api/sessions');

	return response.data.data;
}

And the code to interact with the users API, that signs up users or fetches the currently authenticated user in the session.

import redaxios from 'redaxios';

export async function getCurrentUser(): Promise<User> {
	const response = await redaxios.get('/api/user');

	return response.data.data;
}

export async function signUp(params: {
	email: string;
	name: string;
	password: string;
}): Promise<User> {
	const response = await redaxios.post('/api/user', { user: params });

	return response.data.data;
}

All the methods above throw an error if something happens. Validation errors, wrong passwords, users not logged in, and other things like network errors and such.

Now, on to the context API stuff.

import React, { createContext, ReactNode, useContext, useEffect, useMemo, useState } from 'react';
import { useHistory, useLocation } from 'react-router-dom';
import * as sessionsApi from './api/sessions';
import * as usersApi from './api/users';

interface AuthContextType {
	// We defined the user type in `index.d.ts`, but it's
	// a simple object with email, name and password.
	user?: User;
	loading: boolean;
	error?: any;
	login: (email: string, password: string) => void;
	signUp: (email: string, name: string, password: string) => void;
	logout: () => void;
}

const AuthContext = createContext<AuthContextType>({} as AuthContextType);

// Export the provider as we need to wrap the entire app with it
export function AuthProvider({ children }: { children: ReactNode }): JSX.Element {
	const [user, setUser] = useState<User>();
	const [error, setError] = useState<any>();
	const [loading, setLoading] = useState<boolean>(false);
	const [loadingInitial, setLoadingInitial] = useState<boolean>(true);
	// We are using `react-router` for this example,
	// but feel free to omit this or use the
	// router of your choice.
	const history = useHistory();
	const location = useLocation();

	// Reset the error state if we change page
	useEffect(() => {
		if (error) setError(undefined);
	}, [location.pathname]);

	// Check if there is a currently active session
	// when the provider is mounted for the first time.
	//
	// If there is an error, it means there is no session.
	//
	// Finally, just signal the component that the initial load
	// is over.
	useEffect(() => {
		usersApi
			.getCurrentUser()
			.then((user) => setUser(user))
			.catch((_error) => {})
			.finally(() => setLoadingInitial(false));
	}, []);

	// Flags the component loading state and posts the login
	// data to the server.
	//
	// An error means that the email/password combination is
	// not valid.
	//
	// Finally, just signal the component that loading the
	// loading state is over.
	function login(email: string, password: string) {
		setLoading(true);

		sessionsApi
			.login({ email, password })
			.then((user) => {
				setUser(user);
				history.push('/');
			})
			.catch((error) => setError(error))
			.finally(() => setLoading(false));
	}

	// Sends sign up details to the server. On success we just apply
	// the created user to the state.
	function signUp(email: string, name: string, password: string) {
		setLoading(true);

		usersApi
			.signUp({ email, name, password })
			.then((user) => {
				setUser(user);
				history.push('/');
			})
			.catch((error) => setError(error))
			.finally(() => setLoading(false));
	}

	// Call the logout endpoint and then remove the user
	// from the state.
	function logout() {
		sessionsApi.logout().then(() => setUser(undefined));
	}

	// Make the provider update only when it should.
	// We only want to force re-renders if the user,
	// loading or error states change.
	//
	// Whenever the `value` passed into a provider changes,
	// the whole tree under the provider re-renders, and
	// that can be very costly! Even in this case, where
	// you only get re-renders when logging in and out
	// we want to keep things very performant.
	const memoedValue = useMemo(
		() => ({
			user,
			loading,
			error,
			login,
			signUp,
			logout
		}),
		[user, loading, error]
	);

	// We only want to render the underlying app after we
	// assert for the presence of a current user.
	return (
		<AuthContext.Provider value={memoedValue}>{!loadingInitial && children}</AuthContext.Provider>
	);
}

// Let's only export the `useAuth` hook instead of the context.
// We only want to use the hook directly and never the context component.
export default function useAuth() {
	return useContext(AuthContext);
}

Now this useAuth.tsx file exports both the AuthProvider and the useAuth. In order to use the hook, we need to wrap the entire app (or the parts that need authentication), with the provider.

import React from "react";
import useAuth, { AuthProvider } from "./useAuth";

function InnerApp() {
  const { user, loading, error, login, signUp, logout } = useAuth();

  // Do whatever you want with these!
}

export default function App() {
  return (
    <AuthProvider>
        <InnerApp />
    </AuthRouter>
  );
}

Now I cut the InnerApp content short because I'm going to show you how this would look in a more "production line" environment. We will integrate react-router with this hook in order to create login and sign-up pages, and also add protected routes.

First, let's create two page components, one for signing up users and another for login.

import React, { FormEvent } from 'react';
import { Link } from 'react-router-dom';
import useAuth from '../useAuth';

// Just regular CSS modules, style, however, you desire
import styles from './index.module.css';

// This is a uncontrolled form! No need to manage state for each input!
export default function SignUpPage() {
	const { signUp, loading, error } = useAuth();

	async function handleSubmit(event: FormEvent<HTMLFormElement>) {
		event.preventDefault();

		const formData = new FormData(event.currentTarget);

		signUp(
			formData.get('email') as string,
			formData.get('name') as string,
			formData.get('password') as string
		);
	}

	return (
		<form className={styles.root} onSubmit={handleSubmit}>
			<h1>Sign up</h1>

			{/*
          On a real world scenario, you should investigate
          the error object to see what's happening
      */}
			{error && <p className={styles.error}>Sign up error!</p>}

			<label>
				Name
				<input name="name" />
			</label>

			<label>
				Email
				<input name="email" type="email" />
			</label>

			<label>
				Password
				<input name="password" type="password" />
			</label>

			{/*
        While the network request is in progress,
        we disable the button. You can always add
        more stuff, like loading spinners and whatnot.
      */}
			<button disabled={loading}>Submit</button>

			<Link to="/login">Login</Link>
		</form>
	);
}

Now, the login page.

import React, { FormEvent } from 'react';
import { Link } from 'react-router-dom';
import useAuth from '../useAuth';

import styles from './index.module.css';

// Again, uncontrolled forms!
export default function Login() {
	const { login, loading, error } = useAuth();

	function handleSubmit(event: FormEvent<HTMLFormElement>) {
		event.preventDefault();

		const formData = new FormData(event.currentTarget);

		login(formData.get('email') as string, formData.get('password') as string);
	}

	return (
		<form className={styles.root} onSubmit={handleSubmit}>
			<h1>Login</h1>

			<label>
				Email
				<input name="email" />
			</label>

			<label>
				Password
				<input name="password" type="password" />
			</label>

			<button disabled={loading}>Submit</button>

			{/*
        As I said above, these errors can happen for
        more reasons, like network errors.
        Control these as you desire!
      */}
			{error && <p className={styles.error}>Bad login/password</p>}

			<Link to="/sign_up">Sign Up</Link>
		</form>
	);
}

Finally, let's just add a very simple home page so users go somewhere after logging in.

import React from 'react';
import useAuth from '../useAuth';

import styles from './index.module.css';

export default function HomePage() {
	const { user, logout } = useAuth();

	return (
		<div className={styles.root}>
			<p>Hello {user!.email}</p>

			<button type="button" onClick={logout}>
				Logout
			</button>
		</div>
	);
}

Now, let's revisit the root of our app. We are going to use react-router-dom to build the routes of our app, and we are also going to add a way to protect routes so that only logged-in users are able to access them.

import React from 'react';
import { BrowserRouter, Switch, Route, RouteProps, Redirect } from 'react-router-dom';
import useAuth, { AuthProvider } from './useAuth';
import HomePage from './HomePage';
import LoginPage from './LoginPage';
import SignUpPage from './SignUpPage';

// As the router is wrapped with the provider,
// we can use our hook to check for a logged in user.
function AuthenticatedRoute({ roles, ...props }: RouteProps) {
	const { user } = useAuth();

	if (!user) return <Redirect to="/login" />;

	return <AsyncRoute {...props} />;
}

function Router() {
	return (
		<Switch>
			<AuthenticatedRoute exact path="/" component={HomePage} />
			<Route exact path="/login" component={LoginPage} />
			<Route exact path="/sign_up" component={SignUpPage} />
		</Switch>
	);
}

export default function App() {
	return (
		<BrowserRouter>
			<AuthProvider>
				<Router />
			</AuthProvider>
		</BrowserRouter>
	);
}

Now you have protected routes that redirect anonymous users to the login page!

Wrapping up

Hope this is useful for you! This is pretty close to the scenarios we have in production, but the bulk of the logic is all here. Add some robust error handling and you are all set!

Please check our Phoenix/React starter project if you want to see this in action. The code is not 100% what you see in this tutorial, and might change as time goes on and our requirements change, but it's always going to be a great starting point with authentication already handled.

Stay safe 👋

Next.js and Airtable: a tale of empowering no-code tools

Francisco Sousa — Mon, 07 Dec 2020 00:00:00 GMT

Originally posted on Finiam's blog.

Back in June, we received a request to help build a website for an academic non-profit organization (our friends at AAUM), in Braga, Portugal. They needed a website to help spread awareness of the MinhoCovid19 movement, a group of people trying to supply organizations in need (nursery homes, hospitals, etc) with protective materials built by volunteers.

Their operation was relatively simple, in theory at least. They connected donators of either money or materials, to the organizations in need, handling all of the logistics. The volunteers were using Airtable to manage inventories, deliveries, and also the transparency reports regarding financial movements, which is required to be public by Portuguese laws.

Getting technical

What they needed was a simple marketing landing page, with some stats that represented the actual progress of the movement. The team at the time was hoping that we could set up a CMS to power the web site's data and copy, and they would copy over manually some of the data from Airtable to said CMS. Unbeknownst to them, Airtable could solve most of these problems directly, because Airtable itself would generate HTTP APIs for each of the tables they had, and we could directly fetch the data from there.

So, we decided to build the website with Next.js, as most of the developers that volunteered to help had React experience. Next.js static site generation abilities were also a factor in choosing it, as we didn't want to manage (and pay for) dedicated hosting. We still plugged in Forestry, a git-based CMS, so the team could edit the website's copy easily. Then we would just deploy everything from Github to Netlify. As the site is static, costs would be non-existent or pretty low, and we would not be rate-limited by Airtable at all.

Show me the code!

Disclaimer: not all of the Airtable data could be used on the Website. Currently, the transparency report is being imported from the Airtable, the rest is managed on the CMS side. We are working with the volunteering team to migrate every single data piece to Airtable (except the copy of the website, which will be kept on Forestry).

With Next.js it's pretty easy to integrate with Airtable. First, install Airtable:

yarn add airtable

// or

npm install airtable

Then we just need to get an API key on Airtable, you can follow this guide on how to get there but just look for it on your account settings. I recommend that you create an Airtable bot user, a separate account with access just to the base you need. Each base is basically a project on Airtable so, that way, I avoid using my own API key. If an evil thief gets access to it, all of my bases would get compromised. To even make it more secure, we created a user that only has read-only access to the respective base.

After getting the API key, just create a .env file with this content:

AIRTABLE_API_KEY=api_key

Next.js will automatically pick up that value from your .env file.

Now, let's see the content of our table.

The table labels are in Portuguese, translated to English they are: ID, Purpose, Date, Value. Basically, they represent a list of financial transactions, with their purpose (whether they were donations, material acquisition, etc), their date, and their value.

Airtable automatically generates an HTTP API for each table. You can play with it on their API playground and figure out how to use it. They can even generate Javascript code for listing, retrieval, and create operations. It acts as a good base for what we want to do next.

Now, we want to get all of these values on our Next.js app. In our case, we slightly changed the generated code to do what we needed. This is how we did it.

const Airtable = require('airtable');

const base = new Airtable({ apiKey: process.env.AIRTABLE_API_KEY }).base(
	'<HIDDEN BECAUSE SECURITY STUFF>'
);

export default function getTransparencyReport() {
	const totalRecords = [];

	return new Promise((resolve, reject) => {
		base('Relatório de Transparência')
			.select({
				fields: ['Propósito', 'Data', 'Valor'],
				sort: [{ field: 'Data', direction: 'desc' }]
			})
			.eachPage(
				function page(records, fetchNextPage) {
					records.forEach((record) => {
						const id = record.getId();
						const purpose = record.get('Propósito');
						const date = record.get('Data');
						const value = record.get('Valor');

						if (!purpose || !date || !value) return;

						totalRecords.push({
							id,
							purpose,
							date,
							value
						});
					});

					fetchNextPage();
				},
				function done(err) {
					if (err) return reject(err);

					return resolve(totalRecords);
				}
			);
	});
}

We extract the fields from the table, sorted by date, and return an array of objects with these keys [date, purpose, value]. Then, we use Next.js data fetching mechanism getStaticProps to get this data at build time.

import React from "react";
import PropTypes from "prop-types";

import getTransparencyReport from "root/lib/getTransparencyReport";

// Excluded most of the code for simplicity sake

export default function TransparencyPage({ transparencyReport }) {
  return <>
    {transparencyReport.map(reportLine => (
        <div key={reportLine.id}>
            <p>{reportLine.date}</p> &nbsp;
            <p>{reportLine.purpose}</p> &nbsp;
            <p>{reportLine.value}</p>
        </div>
    ))}
  </>;
}

TransparencyPage.propTypes = {
  transparencyReport: PropTypes.arrayOf(
    PropTypes.shape({
      id: PropTypes.string.isRequired,
      date: PropTypes.string.isRequired,
      purpose: PropTypes.string.isRequired,
      value: PropTypes.number.isRequired,
    })
  ).isRequired,
};

export async function getStaticProps() {
  return {
    props: {
      transparencyReport: await getTransparencyReport(),
    },
  };
}

I've omitted most of the code here for easy demonstration purposes. Check the actual real code at our git repo.

So, every time we run next build && next export we are going to build the entire site and fetch everything from Airtable.

Updating Airtable data

Now, we are using Next.js as a static site generator. The website data is not live but updates are made daily if not weekly at best. So how do we update the data on Airtable and trigger a site rebuild? If we make changes on the CMS, we trigger git commits to the repo and Netlify picks that up and rebuilds the site.

However, Airtable does not have any sort of notification mechanism (like webhooks for example) to trigger Netlify builds (at the time of writing this blog post). The only option that's left is to schedule periodic builds.

We decided to settle on 1 periodic build per day, using Github Actions to deploy everything.

To start building the website on Github Actions, just add the necessary environment variables to your Github project's secrets section. Also, set these secrets on your repository settings.

NETLIFYSITE_ID - Go to _Site settings > General > Site details > Site information, and copy the value for API ID.
NETLIFYAUTH_TOKEN - Go to _User settings > Application > New Access Token
AIRTABLE_API_KEY - you can use your local AIRTABLE API key

Now, we need to define the workflow.

name: Daily Netlify Deploy

on:
  schedule:
    - cron: '0 0 * * *'

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v1

      - uses: actions/setup-node@v1
        with:
          node-version: ${{ matrix.node-version }}

      - name: Build website
        run: yarn && yarn build
        env:
            AIRTABLE_API_KEY: ${{ secrets.AIRTABLE_API_KEY }}

      - name: Upload to netlify
        uses: netlify/actions/cli@master
        with:
            args: deploy --prod
        env:
            NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
            NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}

We are using the schedule option to trigger this workflow every day at midnight. Then our steps are very simple, we just run our build script, and use the netlify-cli action to deploy the website with the prod flag, which will actually overwrite the existing regular Netlify build with the new one.

If your project is open-source, Github Actions minutes are completely free. Also, you do not consume Netlify build minutes if you already upload a finished build. I use this pattern on pretty much all of the static websites I build with periodic rebuilds, to keep their data up to date.

Final notes

This is another example of how no-code tools are not here to put us, developers, out of a job. They have a lot to offer to us via cool integrations like these ones. It also shows that services with good developer docs and APIs, like Airtable, end up being a massive success.

Feel free to check the Github repo for inspiration for your projects.

Stay safe out there ❤️

Reactive Rails applications with StimulusReflex

Francisco Sousa — Wed, 30 Sep 2020 00:00:00 GMT

Originally posted on Finiam's website.

A while ago I made a blog post about the modern web with just Rails (and a few other things). Today I'm going to explore another way of doing awesome things with Rails, in the spirit of the modern, reactive, and real-time, web. Let's explore StimulusReflex, an extension to the amazing library made by Basecamp, to make server-side reactive applications.

So what is this Stimulus Reflex thing?

StimulusReflex is a Rails library that allows developers to make reactive, real-time apps easily. Much like Phoenix LiveView on Elixir land, this library gives us tools to make reactive UIs running mostly on the server-side.

It uses ActionCable, and their own CableReady extension, to trigger DOM updates from the server-side. It uses WebSockets, so it's faster than using regular HTTP requests with say, Turbolinks, or just consuming a regular API. Imagine kinda like Turbolinks, but sending HTML via WebSockets, instead of regular requests.

StimulusReflex adds the ability to write reflexes. Pieces of code that execute asynchronously and trigger a DOM update. Imagine a React component and some callback that triggers a setState call. These reflexes are fully server-side, so you can read directly from the database and do all kinds of things you would do on a typical Rails controller. You then attach DOM events to these reflexes with data attributes, kinda like Stimulus. You can also call these reflexes from Stimulus controllers directly.

Let's dive right into the action. Imagine a view that renders a group of todo records. Then we want each todo to have a delete button that removes that todo without refreshing the webpage.

First, the controller:

class TodosController < ApplicationController
  def index
    @todos = Todo.all
  end
end

The respective view:

<% @todos.each do |todo| %>
  <div>
    <p>
      <%= todo.description %>
    </p>

    <button
      data-reflex="click->TodoReflex#delete"
      data-todo_id="<%= todo.id %>"
    >Delete</button>
  </div>
<% end %>

Then our reflex becomes:

class TodoReflex < ApplicationReflex
  def delete
    todo = Todo.find(element.dataset[:todo_id])

    todo.destroy
  end
end

You use the element variable, available on all reflexes, to get the DOM element where we define our data-reflex (our button). Then we get our data-todo_id from it.

We then find the todo on the database and delete it. When the method returns, StimulusReflex uses the current controller action to re-render the page with the new info. Then, the new HTML is sent via WebSockets and merged with the previous DOM contents.

You could replace this functionality with a simple remote: true Rails form, but as this is submitted with Websockets, the UI update is much faster and gives user's faster feedback, making the app feel snappier.

How the magic happens

As I said earlier, StimulusReflex uses CableReady to trigger DOM updates from the server-side. It's a library developed by the same authors, and you can use it to do some more low-level stuff. It essentially sends DOM instructions through ActionCable, which is a WebSocket library built into Rails.

The magic happens in those reflex classes. When you trigger a reflex, StimulusReflex essentially rebuilds the current page, calling your controller action again, and it sends that new HTML via the ActionCable socket. Then, the client-side StimulusReflex library morphs the updated DOM with morphdom. By default, it updates the entire page, but you can reduce the scope of the updates manually to increase performance, check their docs for more info.

This is a very quick primer on this amazing library. Their docs explain all of this in detail, plus some advanced use cases. It also contains all the instructions to deploy this to production without slowing down the entire application!

The classic todo app

So let's revisit that example I showed you earlier. That example called reflexes directly on the markup. It's a quick way of doing it, but the best way is through a Stimulus controller itself. It gives you more control, allowing you to do cooler things.

The objective here is to make a simple todo list that allows you to create and delete todos. We are doing this entirely on StimulusRefex to achieve instant feedback without any full-page refreshes whatsoever. So, the only thing our Rails controller does is rendering the page.

By the way, I'm assuming you already made the Stimulus and StimulusReflex setup steps, and that you have a model and associated database migration, for a Todo with a description text field.

So, our markup then becomes:

<div class="Todos" data-controller="todo">
  <% @todos.each do |todo| %>
    <div class="Todos-row">
      <p>
        <%= todo.description %>
      </p>

      <button
        class="Todos-delete"
        data-action="click->todo#delete"
        data-todo_id="<%= todo.id %>"
      >Delete</button>
    </div>
  <% end %>

  <%= form_with id: "todo-form",
        model: @todo,
        class: "Todos-row",
        data: {
          action: "submit->todo#create",
        } do |f|
  %>
    <%= f.text_field :description %>
    <%= f.submit "Add todo" %>
  <% end %>
</div>

The important here is the data attributes. Firstly, we annotate the top-level div with data-controller="todo" which tells Stimulus that it should attach the todo_controller.js to this markup. Then we just define two actions. click->todo#delete on the button, to delete todos, and submit->todo#create to submit the form and create todos. Notice that on the delete button we also set data-todo_id="<%= todo.id %>" to so the reflex knows what todo it should delete.

Now, our Stimulus controller:

import ApplicationController from './application_controller';

export default class TodoController extends ApplicationController {
	async delete(event) {
		event.preventDefault();
		event.stopPropagation();

		this.showLoading();
		await this.stimulate('Todo#delete', event.currentTarget);
		this.hideLoading();
	}

	async create(event) {
		event.preventDefault();
		event.stopPropagation();

		const form = event.currentTarget;

		this.showLoading();
		await this.stimulate('Todo#create', event.currentTarget);
		this.hideLoading();
		form.reset();
	}

	showLoading() {
		document.body.classList.add('wait');
	}

	hideLoading() {
		document.body.classList.remove('wait');
	}
}

Here we can see the StimulusReflex stimulate method. It allows you to call reflexes from Stimulus controllers. We pass the event.currentTarget so our reflexes have access to the DOM element where the event handler was specified (the button and the form).

We also have two methods to show a little loading spinner! The stimulate method returns a promise, so you can do things before and after and keep everything in sync. We also reset the form after submitting it, which doesn't happen automatically because we are canceling the default events so we don't submit a real POST request.

Finally, our reflex:

class TodoReflex < ApplicationReflex
  def create
    Todo.create(todo_params)
  end

  def delete
    todo = Todo.find(element.dataset["todo_id"])

    todo.destroy
  end

  private

  def todo_params
    params.require(:todo).permit(:description)
  end
end

Our delete function remains unchanged, but notice the cool thing about the create. Because the DOM element that triggered the event is a form, we can use the classic Rails param mechanisms, so the code looks like a classic Rails controller.

This is everything you need to render a list of todos and also delete and create them as you wish. The styling with CSS is not part of this blog post scope, I'll leave that up to you.

Caveats

The basic setup of StimulusReflex won't cut it for most Rails apps. They have an authentication and a deployment section on their docs that explain the major caveats.

But, in short, if your app has authentication, you need to identify each ActionCable connection with the currently logged in user, otherwise, the users will see each other's updates because they will all share the same socket.

Also, you need to configure your app to use Redis as a cache on a production environment. I recommend doing this in all environments. It's a good practice for your dev environment to be as similar as possible to your production environment.

Finally, if you are worried about Rails' ability to scale, just use AnyCable. It boosts Rails' ActionCable by delegating socket management to a piece of software written in Go which is miles better than Ruby for this workload. Rough benchmarks show that a Rails instance with ActionCable might handle 4000 connections while a single AnyCable node might handle 10000.

Wrapping up

There you go, another cool way of making reactive interfaces without SPAs. It's a very nice manner of adding bits of reactivity to existing Rails' apps.

We are using it for internal projects, trying to add some interactivity on some classic Rails apps.

I hope this has been useful in some way, it's not a detailed tutorial but more of a showcase of another interesting out-of-the-box technology. If you are interested in this sort of thing (alternatives to SPAs) but are afraid of jumping into Ruby because of performance concerns (an overrated concern), checkout Phoenix LiveView. It's written in Elixir, which runs on the BEAM. Performance will not be a problem whatsoever, and... we might have a blog post coming about that soon ;)

Spicing up your Rails frontend experience

Francisco Sousa — Wed, 19 Aug 2020 00:00:00 GMT

Originally posted on Finiam's website.

Rails is great. You can still check DHH's 15-minute blog demo and just appreciate the combination of features and ease of use of the framework. Even though there are even easier ways to make a blog nowadays (cough cough Gatsby), Rails is still a rock-solid choice for crafting digital products.

Rails and the modern web

Despite being a Rails advocate, I mainly used it to build APIs, resorting to React for the frontend stuff. Working with some demanding web designers, I felt that React always had a flexibility that I did not get with HTML and JS. Also, having worked on very large codebases for both full Rails with HTML and Javascript (jQuery at the time) and Rails with a SPA on top, the latter always evolved better in terms of technical debt and time needed to implement features.

There are pros and cons to each approach and we like to analyze those on a per-project basis. More and more we see great success on teams going with fully server-side approaches, like Basecamp for example. Github is another case of a mostly server-side Rails app that works great (most of the time).

Even with a fully server-side approach, you can create experiences that for end-users, will be indistinguishable from a fully client-side web app. The instant page transitions, components, and cool APIs to interact with the DOM. Let's see how.

Turbolinks and remote helpers

The big deal about SPAs is on its name. single page applications don't have page transitions, it's usually done in the client-side and it looks and feels like a native app. An issue with server-side approaches is that every page change (clicking links) triggers a server request and a subsequent full page load. But you can fix that with Turbolinks.

Perhaps you've heard about it, Turbolinks has been on the Rails ecosystem for quite a while. It basically is a piece of Javascript that detects navigation requests, like clicking a link, and then fetches the page and merges the head and swaps the body. Thus, changing a page without a full refresh.

How do you install it? Simple, you don't, Rails comes with it installed and set up by default! For non-Rails projects it's pretty simple, just check out their documentation

Not only Rails has default Turbolinks support, but it also has a ton of small features to assist in implementing instant user interactions. You can submit forms and trigger controller actions with links and buttons with AJAX and no extra code. Their doc page about working with Javascript has a pretty detailed explanation of these "remote elements".

The only thing that is not handled by default in Rails and Turbolinks is when a controller action renders a new template. Turbolinks supports redirect_to out of the box, but when a template is rendered using render, it causes a full page refresh. This often happens after you submit a form. Even if you are using the form_with helper, which triggers an AJAX request instead of the traditional form submission, you will still re-render the page.

Take for instance, this create controller action:

def create
  @schedule = Schedule.new(schedule_params)
  @statuses = User.statuses.keys

  if @schedule.valid?
    schedule_status_change(@schedule)

    redirect_to(new_schedule_path, notice: "Scheduled!")
  else
    flash.now[:alert] = "Invalid!"

    render action: "new"
  end
end

By default, Turbolinks handles the case where the form is valid because we use redirect_to. But when the form is invalid, we will have a full page refresh, because we are re-rendering with the render method.

Solution? Just install this handy gem made by Jorge Manrubia. That's it. Right now, at the time of writing, Turbolinks version 5 doesn't support this, but the upcoming version 6 will handle this with no extra setup.

Components

Right now, developers and designers on the web design and implement UI with a component-based approach. It's easier to organize and re-use code this way. It's also the way design systems are usually maintained. And now with design systems becoming more and more popular, components even feel like the way to make web applications.

The traditional SPA assumes this model, usually, and it's pretty easy to do it. However, on Rails, there is no notion of this, and you are on your own to implement your components. The most popular way is just to use CSS. Naming schemes like SuitCSS and BEM allow you to define reusable CSS classes for your project. But what happens when you also need markup or behavior?

You can always implement your components using view partials, and your behavior as regular Javascript files, but that isn't ideal. What if there was a way to encapsulate CSS, JS, and HTML on a nice package, and re-use that throughout your app?

Introducing view_component, made by Github. Using this library, you can define all of your components on app/components.

A component is a combination of a Ruby class, an ERB template, a CSS file, and a JS file. You can control server-side behavior on that Ruby class, define your markup on your ERB template and then control the frontend side of things with CSS and JS.

As an example, I'll show you the implementation of a flash alert component, that we can use to render Rails flash notices and errors.

First, we need to define the Ruby class that renders the component. In its basic form, it's just a regular constructor that you use to set instance variables, kinda like a controller.

The render? method can be used to determine if we should render the markup of the component. In this case, we don't render anything at all if we don't have any flash messages.

class FlashMessageComponent < ViewComponent::Base
  def initialize(flash:)
    @flash = flash
  end

  def render?
    @flash&.keys&.count&.positive?
  end
end

Then we just define markup like we would for a regular Rails view. You can use any template engine you like (slim for example). I prefer traditional ERB:

<div>
	<% @flash.each do |key, value| %>
	<div class="Flash Flash-<%= key %>"><%= value %></div>
	<% end %>
</div>

Some basic styling:

.Flash {
	position: fixed;
	top: 12px;
	left: 50%;

	display: inline;
	padding: 0.5rem;

	color: white;

	transform: translateX(-50%);
}

.Flash-notice {
	background-color: cadetblue;
}

.Flash-alert {
	background-color: red;
}

And that's all it takes to define a component. To use it, we just reference it like this:

<%= render(FlashMessageComponent.new(flash: flash)) %>

You can do this for all sorts of UI elements and it allows you to encapsulate UI logic on these re-usable bits. Now, you probably noticed that we don't have any JS here. Will touch that in a bit, let's just talk about Stimulus first.

Stimulus

Now, our previous example was pretty nice, but we lack JS to make the flash component behave. When I first started working with Rails (back in 2014) the defacto way was to just use jQuery to make stuff all interactive and nice. And we all know how that ends up with large codebases.

Spaghetti code get it? Yes, it's a lame joke.

The folks at Basecamp, who pretty much invented Rails anyway, created this simple library (not a framework) to solve the mess of having a bunch of random Javascript with jQuery all over the place. Stimulus uses simple data attributes and small Javascript controllers to add rich interactions to Rails apps (also works without Rails).

I'm not going super in-depth on Stimulus, their docs are pretty thorough and easy to pick up. Still, let's see how we can combine Stimulus with view_component to make our flash component nicer. Currently, when a flash message is rendered, it stays there forever, so lets us make it go away after some time and also add a button to dismiss it.

Refer to Stimulus docs to set it up. Then refer to this part of the view_component docs to integrate both libraries. By default Stimulus just searches for controllers on the app/javascript/controllers path. With these integrations, we can also define controllers on the app/components folder.

Now, back to our flash component. Let's change our old markup a little bit. We need to add the button and the respective Stimulus related attributes.

<div data-controller="flash-message">
  <% @flash.each do |key, value| %>
    <div class="Flash Flash-<%= key %>">
      <%= value %>

      <button data-action="click->flash-message#close">Close</button>
    </div>
  <% end %>
</div>

We added the data-controller attribute, to tie this markup with its respective Stimulus controller. Then we added a button with a data-action attribute. We specify click->flash-message#close, which means we are mapping the click event with the close function of our flash-message Stimulus controller.

The naming of these files matters a lot, like most Rails stuff. The flash_message_controller defines the flash_message controller. If you want to use it on your markup, you must reference it that way, always. And yes you can use it in different places on your HTML. Warning, controllers on app/components can have name collisions with components on app/javascript/controllers.

Now, the controller:

import { Controller } from 'stimulus';

export default class extends Controller {
	connect = () => {
		this.timeout = setTimeout(() => this.close(), 5000);
	};

	disconnect = () => {
		clearTimeout(this.timeout);
	};

	close = () => {
		this.element.remove();
	};
}

Stimulus controllers have a bunch of lifecycle related functions, you can check those on the docs, but here we are redefining connect and disconnect. We use those to set a timeout to close the flash message and also to clear that message if the controller is unloaded. Just a nice touch.

Then, we get to our close function that we use on the markup. Every Stimulus controller has access to the respective DOM element where it's being used. That data-controller attribute takes care of that. When we use this.element we are reaching for that DOM piece, then we can just call .remove to delete that from the DOM, making the flash message go away.

A fun alternative

Remember, pick technologies that make your team happy and productive! In our case, everyone loves React and everyone loves Rails, but it's refreshing to just go full Rails once in a while and break out of the norm. If you are interested in solving problems more on the server-side, keep tuned, as we are going to talk about StimulusReflex for Rails and also LiveView for Phoenix.

The modern web is great because you have a ton of choices on how to do things, let's cheers on that.

Building and hosting a Discord bot

Francisco Sousa — Thu, 30 Jul 2020 00:00:00 GMT

Discord is one of the premier instant communication platforms on the web right now. Free, easy to use, and just cool, Discord is one of the main hangout apps for gamers, developers, and just overall tech people. One of the coolest things about it is the ability to install bots on your servers that make some cool stuff. Bots that play music, text-based RPGs, dice rollers for RPGs (I made one), meme generators, and many more. In this blog post, I'm going to walk you through on all the steps needed to make your bot and host it for free (kinda free), provided you have some basic ruby and git knowledge.

Getting started

Why Ruby? Well, no reason at all. It's just my favorite language at the moment of writing. Its also the language I have been using at work for these past few years. And, I find it's just so simple and easy to understand for people who use it sparingly.

And why Heroku? It's a hosting platform that integrates with Github and automatically provides a CI/CD pipeline that continuously deploys your changes. This way you can make changes to your bot, commit that with git and your bot will just update itself on Heroku. We just need a little hack to make it work there but will get to that.

So some prerequisites:

Discord account
Github account
Heroku account
Basic Ruby knowledge
Ruby (latest version) and Git installed and available on your path

After you create all of these accounts, make sure you create a new Discord app. For that visit their developer portal. Click New Application on the top right corner and follow along.

After creating it, on the bot's main page you can take note of the CLIENT ID, we are going to need it. Then go to the Bot tab on the side menu and. On that new screen, you can add a bot user to your application. Do it and take note of the TOKEN value also. All the other options are not needed for now, but you'll probably explore those later on as you develop your bot.

The bot

Now make sure your directory dedicated to this project and also don't forget to initialize a git repo, we are going to need it. Now, like most ruby projects, let's start with a Gemfile.

source "https://rubygems.org"

ruby "2.7.1"

gem "discordrb"
gem "dotenv"
gem "rake"
gem "zeitwerk"

After creating and saving this file, run bundle install to install these dependencies.

Now, what do these gems do? Let's go through them:

discordrb - this is a wrapper of the Discord API. We are going to need this to effectively communicate with Discord
dotenv - we are going to use this to load the secret keys that we got from Discord
rake - the classic ruby task runner. We are going to use this to start the app.
zeitwerk - a class loader. We'll set up this soon.

Now, let's write the code that will connect to discord and handle incoming commands. Let's place it on src/discord_bot.rb.

module DiscordBot
  throw "Lacking required secrets!" unless ENV["TOKEN"] && ENV["CLIENT_ID"]

  @bot = Discordrb::Bot.new(
    token: ENV["TOKEN"],
    client_id: ENV["CLIENT_ID"],
  )

  puts "This bot's invite URL is #{@bot.invite_url}"
  puts "Click on it to invite it to your server"

  @bot.message(with_text: "!ping") do |event|
    event.respond "pong"
  end

  def self.run
    @bot.run
  end

  def self.invite_url
    @bot.invite_url
  end
end

We use the discordrb library to create a bot instance using the values from our shell environment.

Then, we define a handler for a bot command. The handler will intercept every message that matches !ping and respond to it with a pong message.

Then we just declare methods to run the bot and get the invite URL. The bot run method connects to Discord via a persistent websocket connection. That's how our bot reads all of the incoming messages from the servers it's installed in.

To correctly set the environment, create a file called .env and add the values you got on the Discord developer portal. The dotenv gem will take care of loading all of those values into the environment, just when we run our application. On Heroku we will define these variables on their dashboard. But for now, the .env file.

CLIENT_ID=the client id you got on discord dev
TOKEN=the client secret you got on discord dev

DO NOT FORGET TO ADD THIS FILE TO THE GITIGNORE OF YOUR PROJECT otherwise, you are leaking secret information to the public.

Now to wrap everything up, let's write our Rakefile so that we can run our discord bot.

ENV["RUBY_ENV"] ||= "development"
require "bundler/setup"
require "dotenv/load" if ENV["RUBY_ENV"] != "production"

Bundler.require(:default, ENV["RUBY_ENV"])
loader = Zeitwerk::Loader.for_gem
loader.push_dir(File.dirname(__FILE__) + "/src")
loader.setup

task :bot do
  DiscordBot.run
end

We wrap the entire file in a conditional that checks if we are running the file with ruby and not requiring it from another ruby file. This is usually done on ruby scripts that are meant to be executed from the command line.

Then we require the bundler/setup, so we can automatically require all the gems into the project. We then require dotenv/load only if we are outside of a production environment. During production, we don't want to load environment variables from the .env file, we just want to load them from the Heroku configuration.

We also set up zeitwerk, which will automatically require our own files. In this case, we have a src/discord_bot.rb file which defines a DiscordBot module. Check this link for more info on Zeitwerk file structure. We push the src dir as a root dir for our code and then we run setup to load it.

Then we define the task to run the bot. Then, to start it, just run bundle exec rake bot. Now use the invite URL that shows up in the console and try it on your server! The bot should appear on the online user list and it should respond to a !ping message with pong.

How to put this online

Now the bot works great and it's even ready for deployment. You host this on any server that you might have handy. Just send this code over and run rake bot. But, most of us don't have handy servers there. This is where Heroku comes in.

Heroku is a hosting platform for web apps. Web apps is an important term. Meaning that Heroku expects that your app is an HTTP server of some sort, that binds to a port and responds to incoming request. If you deploy our code to Heroku, it will work for a couple of minutes before shutting down. This is because Heroku shuts down any app that does not bind to any port! So we just need to add a webserver to it.

Sinatra

We are going to spin up a very simple Sinatra webserver that will have a simple HTML page that will contain the invite URL for our bot. It's pretty useful stuff. You just share the Heroku app URL with your friends and they'll be able to invite the bot to their servers.

Let's go back to our Gemfile. Just the add gem "sinatra" to it and then run bundle install again.

Now let's set up our webserver. We need to create a folder at src/webserver and then one more subfolder at src/webserver/views. The views folder will hold all of our HTML templates. In this case, just one.

Then we create our HTML template at src/webserver/views/index.erb.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>The greatest bot</title>
    <style>
      body {
        margin: 40px auto;
        max-width: 650px;
        line-height: 1.6;
        font-size: 18px;
        color: #444;
        padding: 0 10px;
      }
    </style>
  </head>
  <body>
    <h1>Greatest Bot</h1>

    <p>
      I am the greatest bot. Add me to your server with my glorious
      <a href=<%= @url %>>invite url</a>.
    </p>

    <p>
      This is a good place to post instructions on how
      to use your bot by the way!
    </p>
  </body>
</html>

Then create a src/webserver.rb file and a src/webserver folder. On our src/webserver.rb we are going to define the homepage route of our web app.

class Webserver < Sinatra::Base
  set :views, File.dirname(__FILE__) + "/webserver/views"

  get "/" do
    @url = DiscordBot.invite_url

    erb :index
  end
end

When we receive a request on the / route, we set the @url with the bot's invite URL. Then we render the index.erb.

Finally, we just need to add another task to our Rakefile file to boot up the webserver and the bot at the same time. Add this to the Rakefile.

task :web do
  Rack::Server.start(
    Port: ENV["PORT"] || 4567,
    app: Webserver.new,
  )
end

task :all do
  Process.fork { Rake::Task["bot"].invoke }
  Process.fork { Rake::Task["web"].invoke }
  Process.waitall
end

The web task boots up a webserver that listens for requests at the port defined on our environment or 4657. By default, Heroku adds a PORT variable that holds the port we need to bind to. On the other hand, the all task spawns two processes that will run at the same time. One for the webserver and other for the Discord bot itself. This will allow us to run the entire thing for free, more details on that later.

During development, you can always just work on the bot or the webserver, or both at the same time.

The final step we need is to create the Procfile. On this file, we specify what Heroku will run. In this case, we want Heroku to run everything in one dyno, the web dyno. Heroku can run multiple dynos of different kinds. We could even have separate dynos, one for the bot and other for the webserver, but to fit on the free tier, we can only use one without running into limits.

web: rake all

Now, to run the bot and the server at the same time do rake all. This will start both scripts and it will display its outputs on different colors. Very useful for debugging and seeing what's happening. Oh, and you can visit your shiny website at http://localhost:5000.

Remember, that with this setup you can always run just the bot or just the webserver locally. rake bot or rake web will do the trick. Just in case you don't want to work on the entire thing.

Heroku

Now, to deploy all of this to Heroku make sure you have all of this code on a Github repo. Then create a new app on Heroku and use the Connect with Github function. Then just follow to connect with your Github repo. Finally, click the Deploy branch button to deploy your code. You can optionally enable automatic deploys so every time you commit to master, you trigger a new build.

Your app will probably not work right away. You need to set the correct environment variables for it to work. Go to the Settings tab on Heroku. Then on the Config Vars zone click reveal and then introduce the following variables:

RUBY_ENV=production
CLIENT_ID=your client id from discord dev portal
TOKEN=your secret token id from discord dev portal

You can use the same values you have on your .env file for the discord secrets. But note that is a best practice to have different apps and different values for the development and production build. If your bot is used by many people, it's probably a bad idea to use the same bot app during development, because it's already running an instance on Heroku and it might introduce downtime for its users. But all it takes is just creating another app as we did earlier, and use those new credentials here on Heroku.

After setting the environment variables, Heroku should automatically re-release the app. Click the Open App in the top-right area. It should take you to the website with the invite URL. Use it for your server, then test the bot as you did during development. By the way, the server on your own machine, in case you are using the same credentials for both production (Heroku) and development, just to make sure there are no interferences.

Disclaimer

Heroku offers a hobby tier that allows you to host an app for free, provided it runs only a few hours per day. Heroku automatically kills your app after it idles for a while. That's based on web requests, so even if people are using the bot on Discord, that doesn't count as traffic for Heroku. Our bot uses websockets to send and receive messages from Discord, so that doesn't count for Heroku. More info on their docs page about the free tier.

Though, if you add a credit card to your account to validate it, you get enough hours to run a single dyno the entire month 24/7. By fitting all of our tasks into the rake all task, we can just consume hours in a single dyno. Not ideal, but it works for our use case.

Next Steps

Now the bot is up running, but it doesn't do much. The hardest part of getting it up and running on the outside world is done, but now it's up to you to fill the rest in.

You should take a look at discordrb wiki page and also their API documentation for more advanced stuff. Their API is plenty easy to work with.

You may also see a "real" bot in action using this template and these libraries. I made a bot a while back to roll dice. I use it just for fun and some online Dungeons & Dragons sessions. You can check the code here on my Github repo and you can also invite it to your server by visiting it's home page.

A final note

This will work for hobby projects of sorts, but any bot with a significant amount of servers will perhaps need a little more horsepower to run, and the hobby tier from Heroku won't cut it. Also, running two processes on one dyno is not the best way to go about a performant. But hey! We usually say that "premature optimization is the root of all evil", so don't sweat it. When things get slow, just split off the webserver and the bot in different dynos and boost them (that won't be free anymore though).

I packaged the teachings of this blog post into a starter repo that you can use instead of following these steps. I'll keep that repo updated but there might be the chance that the code is slightly different. People might find better ways to do things and I'll happily accept that! Anyway, the repo's README will get you sorted. Check it at my Github repo.

Have fun!

Let’s slim down on Javascript

Francisco Sousa — Thu, 23 Jul 2020 00:00:00 GMT

Originally posted on Finiam's website.

The ancient narrative of "Javascript bloat" is ever-present in the tech world. Web developers love Javascript, backend developers hate it and end-users usually don't give a damn about it as long as websites open fast, work as they should, and do not spy on them. Still, web pages have way too much Javascript, especially SPAs, that could be avoided with just a few tips. Let's see how.

Why care?

Nowadays Internet connections are pretty fast (usually) so why worry if your app has 1 megabyte of JS? Most native mobile apps go into the hundreds of megabytes!

So, when dealing with Javascript, you are dealing with a scripting language (it’s in the name you see), so the code needs to be read and interpreted by something. In this case, it’s your own browser. Then that code gets loaded into memory and your browser starts doing stuff with it. The problem here is that it takes time and processing power. So if you have a simple news website, and everyone needs to download 4 megabytes of scripts and trackers to be able to see a news article, you probably just lost a reader. Those 4 megabytes of Javascript would take an immense amount of time to be downloaded, read and parsed, especially on mobile devices. Despite the awesomeness of our internet connections, most people access it via WiFi or cellular networks (3G, 4G, etc). These are unreliable and slow even in areas with good coverage. Also, note that the average phone is not that fast.

The more Javascript you use, the more it takes to fetch a page from the server, the more it takes to render the content, and in the end, the more it takes the user to view the webpage. Even with top-class server tech and CDNs.

A good starting point is measuring your website performance with Google Lighthouse (the Audits tab on Chrome’s Web Tools). Metrics like Largest Contentful Paint and Time to Interactive are good ways to see if Javascript is slowing down your website loading. WebPageTest is also a great tool that allows you to test your website with different devices, like slow smartphones on a limited connection, so you can better test real-world situations.

Given that, let's see how we can improve our website's performance metrics, with a few tips.

Ditch SPAs

Ok, so this one is a bit extreme and probably only useful before you actually start working on your project. The fastest Javascript is no-Javascript at all. No code is the best code or so they say. By server-rendering your apps you can just ditch client-side Javascript entirely. Using something like Rails, Phoenix, Laravel or any full-stack web framework will allow you to implement web apps entirely rendered on the backend.

I know, shocking. We made apps like this for ages and it kinda worked out. Check out Basecamp, they're doing pretty well. Having worked with multiple SPA libraries and server-rendered full-stack frameworks, I can attest that most times, we duplicate a bunch of concepts on the client-side that could just exist as a full-stack unit: routing, data validations, app state, API calls (when full-stack, these just don't exist).

Working in a digital agency I often saw SPAs applied to very simple projects, where a classic Rails app would excel. Mea culpa, I’ve done that myself. A client-side approach is great when you want to have separate teams for backend and frontend. Having dedicated teams for both will help teams organize themselves. After all, all these SPA frameworks and libraries were made by very large organizations with very large codebases.

But even for smaller projects, SPAs excel on highly reactive applications, and any application that is trying to emulate a native experience, through animations and navigation controls, for example, will also benefit from these modern frontend technologies.

But still, a very good stack for CRUD like apps can be made with Rails and a tiny bit of JS, with Stimulus and Turbolinks. You get the awesome developer experience of Rails, you avoid the dread of full-page refreshes with Turbolinks, then Stimulus helps you write Javascript in a sensible way. No network request handling, no API writing, no manual auth token handling. Just raw productivity.

And if you are looking for reactivity, check StimulusReflex (for Rails) and LiveView (for Phoenix). Reactivity on the server-side.

Finally, take a look at Basecamp, it only uses Rails and a very lightweight Javascript library called Stimulus and good old Turbolinks. For both Mobile and Desktop.

A final friendly reminder on this matter. Pick something that strikes a good balance between maintainability and your team's happiness. Don't switch over to a specific technology or way of doing things just because a random dude on the internet says so! Explore and see what works for you.

Use native libraries

Now back to Javascript. Often developers reach out for a popular dependency before trying and using the native tools the browser platform offers. It's ok, for most of the Internet history the default libraries on the browser were pretty unstable or just didn't work across multiple browser versions. For example, libraries to make network requests are imported into almost every project when the browser already has a powerful tool for that: fetch.

const response = await fetch('/api/validator', {
	method: 'POST',
	headers: {
		'Content-Type': 'application/json',
		Accept: 'application/json'
	},
	body: JSON.stringify(formState)
});

const json = await response.json();

Right now you probably are using something like axios or superagent on your web app, both sitting at 4.4kb and 6.4kb total size respectively, at the time of writing this blog post. I resort to axios all the time due to old habits, but I've been replacing it with fetch and life has been great.

Most problems pointed to fetch are its lack of defaults and weird error handling (they only throw errors on network failure and not on bad responses), but that can be fixed with a simple custom wrapper. Check out this awesome blog post from Kent C. Dodds where he builds a custom wrapper to remove all problems regarding the lack of sane defaults.

If you want to keep around the axios API, you can always use redaxios. An axios compatible API that uses fetch under the hood and it's just 800 bytes!

And if you need to support older browsers, use unfetch as a polyfill.

Try and use the browser's native functionality before going after solutions on npm, you will be surprised. Browsers can do awesome stuff nowadays and almost every single functionality can be polyfilled back into older browsers.

Be careful with 3rd party dependencies

Even though the browser is a great platform, it's pretty rare to complete a project without ever using a 3rd party dependency. Even if you maximize the natural power of the browser, there are essential libraries that you will probably need. One thing that happens though, is that people often search for a given library and don't really think about the consequences. Any library you use will increase the total size of your web app. We should be mindful of that.

How? Use a tool like Bundlephobia to inspect the size of a package before using it, and to check if it is tree-shakeable. And what is a tree-shakeable package you might ask? It basically means that most Javascript bundler like Rollup, Webpack, and others will remove the code you do not use from said package.

For example, if you import lodash the entire package will end up on your final bundle. However, you can use the alternative lodash-es, which does the same thing and is tree-shakeable, and you only use the functions you import. As long as you do this:

import { uniqueId } from 'lodash-es';

Remember, try to find the right balance between "reinventing the wheel" or adding another dependency. And when you are looking for libraries to solve your problems, pick one that is small and tree-shakeable.

You can also resort to code-splitting and load polyfills conditionally. I'll show you how in a bit.

Code Splitting

If you are using a Javascript bundler, chances are you have the ability to perform code-splitting. It basically consists of splitting your overall Javascript codebase into different modules. It is usually used to not load your entire application at once. If you have a large web app, it's usually a good idea to perform code-splitting, so your users don't need to download every single piece of Javascript in your app.

For example, if you have a React app with react-router you can perform route-based code splitting. Each different page on your app will have its own module plus a common bundle, which will contain the Javascript code that is common to all different modules. This will reduce the initial load size for each part of the web app but at the cost of having to make a network request every time the route changes.

I am not going in-depth on implementation details, but you can check the react-router docs on the best way of doing this. The important thing to note is that we should only load code that the user needs or will almost surely need in the future.

Popular frameworks on top of their SPA libraries like Next.js (React), Nuxt (Vue.js), and Sapper (Svelte) do this out of the box via code splitting based on-page components. This is a cool way of going about this since you do need to manually implement this yourself.

You can even use this strategy to conditionally load dependencies. In the next example, we are importing some polyfills only if the browser does not support the given functionality natively.

if (typeof IntersectionObserver === 'undefined') {
	await import('intersection-observer');
}

if (typeof Map === 'undefined') {
	await import('core-js/es6/map');
}

if (typeof Set === 'undefined') {
	await import('core-js/es6/set');
}

if (typeof window.requestAnimationFrame === 'undefined') {
	await import('raf/polyfill');
}

Apply this to anything you need. You can use this to load different bundles for mobile or desktop. For different user roles, for example, regular users won't probably need to have the admin dashboard code loaded into their browsers.

Don't support older browsers

Dramatic statement. Nowadays you are probably using babel to transpile your JS code for it to be compatible with older browsers. So every single new feature of the language is then ported back to be supported. If you have IE (Internet Explorer) as a target, then babel will convert every arrow function into a regular function. Transpiled code is longer, heavier, and probably not as optimized as the code you have actually written.

How to solve this? Ditch older browsers. I mean, this might seem ridiculous and counter-intuitive at first but older browsers, mostly IE obviously, are insecure, slower, and just plain worse than the alternative. If a computer runs IE it probably can run either Chrome or Firefox. There are a few cases where this is not possible. Some institutions and companies just don't allow people to update or install applications on their computers, so they are stuck with Windows XP and IE.

If you cut down the number of polyfills you need, and the transformations your code needs to run on the browser, you can save some serious space. You can also create different Javascript bundles, one for modern browsers and one for older ones. You can check if the user runs IE and ship off the polyfilled bundles to them, but for that, you would need a server that is parsing the user agent of the HTTP requests. If you are making a JAMstack app you probably can't parse the user agent string efficiently, and everyone will get the same bundle anyway.

polyfill.io is a possibility, it loads polyfills conditionally, based on your browser’s version.

Remember, you can always review your web app user base and check the percentage of users with older browsers. You can track your user's browser versions respectfully by using a privacy-focused tracker (a bit paradoxical) like Goatcounter. They just collect very basic information that cannot uniquely identify users, respecting their privacy. You will probably notice that you don't have IE users at all (this is the case for the products I've been working on at least).

For the global market share, IE has 3%, but it's a good idea to scan the market and see if it makes sense to be IE friendly. Imagine that your app is a specific tool for Linux people. They won't be using IE at all.

It's a matter of user research like all great products should have. An app for the enterprise financial market would probably need IE. Lot's of people on that field are stuck on Windows XP due to organizational restrictions. For a rad startup idea? Probably no IE users will pop up.

At the very least, make sure your landing page works on IE, then just tell people to upgrade :)

Wrapping up

The rationale is simple. Be mindful of the cost of Javascript (probably the greatest blog post on this matter), and use simple solutions to deal with it. Don’t forget that premature optimization is the root of all evil, but a couple of tweaks early on in a project’s lifetime can make a world of difference.

Also, make sure you test your web experiences on slower devices. I cannot say this enough. Not everyone has a Galaxy S20 or the latest shiny iPhone in their pockets.

It all boils down to the user experience. Make accessible, performant web apps that do just what they are supposed to do, well.

Dealing with webfont loading

Francisco Sousa — Thu, 25 Jun 2020 00:00:00 GMT

The time has come to implement a beautiful website design. It has a great Webfont. So you go to Google Fonts or Adobe's Typekit or you just buy one and self host it. But once you start writing code and previewing its results, you notice one little pickle: the whole website shifts, flashes, and janks all over the place when refreshed or opened from a cold cache. It ain't pretty. Let's see how we can attenuate those effects.

FOIT & FOUT

FOIT and FOUT are acronyms likely to pop up when you search for the issues above. If you haven't heard of them before, this is what they mean:

FOIT - flash of invisible text. When the website has no fonts rendered until the web font loads.
FOUT - flash of unstyled text. When the whole typography flashes from an unstyled state to your web font style.

Examples of FOIT (left) and FOUT (right).

These issues are manifestations of a root problem. What the hell does your browser do while your web font loads?

Let's remember what font-family does:

html {
	font-family: 'MyRadWebfont', Arial, sans-serif;
}

You are telling the browser to use MyRadWebFont first, then Arial and, if none of these are available, any sans-serif font the user's device might have. Arial is a safe bet because it's a font that's included in every major browser.

While the web font is loading, you either get a FOIT or FOUT. It usually depends on the user's browser. A couple of years ago, most browsers applied the FOIT approach, detecting if the website is using a web font and then waiting for it until it becomes available and then swapping the invisible text with the font. Then, some browsers (mainly Firefox) started defaulting to the FOUT approach, to make sure users could see some text while fonts loaded.

At the time, if you wanted control over this behavior, you had to do it yourself with custom Javascript. Nowadays, there's an easy way to do it.

`font-display` to the rescue

font-display is a somewhat recent CSS property that can be used to control the way web fonts are loaded and how their fallback is handled. It has awesome browser support. However, it doesn't work on IE. What a surprise.

The property takes on these values:

block - renders nothing at first, then shows the web font. If the web font loading timeouts, it shows the fallback font. An expected FOIT;
swap - renders the fallback, then swaps when the web font is loaded. An expected FOUT;
fallback - same as swap, but if the web font takes too long to load, it doesn't swap and it keeps the fallback. FOUT if it doesn't timeout;
optional - same as fallback, but the browser decides if the ideal conditions to download web fonts are present. On a slow internet connection, for example, the browser won't download it.

This is the gist of it. You can check out Monica's in-depth explanation, demo, and talk on this website.

Still, the same FOIT and FOUT happen, but now at least it's predictable. And you ensure all users will get the same behavior (as long as their browsers support the font-display).

If you don't use Google Fonts, you might be unable to specify a font-display property. Most font providers give you a CSS file with @font-face declarations that you cannot modify, meaning you cannot add the font-display property to it. This is an issue with Typekit (which I use at work all the time), where you cannot apply the font-display property. If you self host your fonts though, all is fine.

Then the only solution that's left is actually handling this stuff with Javascript. Controversial, but I'm gonna try and spin up a solution with a progressive enhancement mindset. Works great if you have JS, still works as expected without JS.

Smoothing out the FOIT

I've had this idea where we knowingly apply a FOIT, by hiding the entire website until the font is available, and then fading in the entire thing. The regular FOIT behavior is a bit ugly, because you still have parts of the website rendered and then the whole thing janks and becomes visible. With this, we nicely fade in the entire website and avoid layout shifts.

We can use the opacity property for that and then we'll apply a simple transition. All with CSS. Then we somehow apply these styles with JS after the font is loaded.

So, I'll exemplify with what is powering my own website. The source code is available on my Github repo, which is an Eleventy + Webpack project.

First, we need to set up fontfaceobserver, either with your favorite JS package manager or by adding it via CDN. Works on IE with a Promise polyfill. It's a small dependency, just 1.3kb gzipped.

The code:

// if you can't require packages, load it via a CDN
const FontFaceObserver = require('fontfaceobserver');

const htmlElement = document.documentElement;

htmlElement.classList.remove('no-js');
htmlElement.classList.add('has-js');

function applyHasFont() {
	htmlElement.classList.remove('no-font');
	htmlElement.classList.add('has-font');
}

new FontFaceObserver('Muli', 2000)
	.load()
	.then(() => {
		applyHasFont();
	})
	.catch(() => applyHasFont());

With this, we replace a no-js class with a has-js class when the script is loaded. Then, we replace a no-font with a has-font class when we load our font, Muli. We also apply the has-font class after a 2-second timeout, just in case the user's internet is really slow and can't download the font in an acceptable time. The second argument of the FontFaceObserver constructor is the desired timeout in milliseconds. After that time elapses, the load function errors.

Then, on the CSS side of things:

@import url('https://fonts.googleapis.com/css2?family=Muli:wght@200..900&display=swap');

html {
	font-family: 'Muli', Arial, sans-serif;
}

.has-js.no-font body {
	opacity: 0;
}

.has-js.has-font body {
	opacity: 1;

	transition: opacity 0.5s;
}

Also don't forget to add the default classes to the HTML document:

<html class="no-js no-font">
	...
</html>

Notice that we only apply the styles if we have JS. This way, people that (for some reason) have their Javascript disabled, still can see the website. We are passing display=swap to the Google Fonts API, so that on the lack of Javascript, this will fall back to the swap behavior.

Wrapping up

I hope this post has been helpful. Personally, I avoid reinventing the wheel with JS for features that can be handled with just CSS, but in this case, a nice compromise can be reached. It respects all users in terms of connection speed and Javascript capabilities and supports different browsers.

A primer on the JAMstack

Francisco Sousa — Fri, 10 Jan 2020 00:00:00 GMT

Originally posted here on my previous, but very nice, company Aurora Digital.

Our own website is a product of the JAMstack. Perhaps you have heard of it if you are a web developer, and if you are not, then this will probably remind you of that jam that your aunt gave you and you never even tasted it and how it's probably full of mold right now. Nevertheless, the JAMstack is not another boring tech-specific stack. It's a change of paradigm. A shift in how we develop web apps and websites.

The usual web

Websites can be developed in all sorts of ways, but most are made by stitching together some HTML, JS, and CSS (the usual web stuff for those of you without a technical background). You drop a bunch of these files on a web server, and now everything is available on the web. Most times you need some way of representing dynamic content, a web page where the structure is the same but only the content changes. A website for a news agency perhaps will need to have a web page for each news article, something like:

www.awewsomenews.com/news/old-man-wins-the-lottery

(that's not a real URL by the way)

Usually, web servers have custom code to handle this, either written by a developer or enabled by some CMS, like Wordpress for example. When a user visits that URL, the webserver code takes the old-man-wins-the-lottery part of the URL, uses that to fetch the specific news article record, which is usually stored on a database, and builds an HTML page with that.

There, I just told you how most websites work.

This is perfectly fine, however, there are some underlying issues here. Using traditional web servers to serve your websites is usually a bad idea, at least if you do it yourself. The major issue is performance. If you are serving hundreds of thousands of web pages per second, one web server won't be enough.

The new way

So what is the JAMstack? JAM stands for Javascript, APIs, and Markup.

But wait? That's basically what everyone nowadays uses. Yes, but... everything is served via static HTML files. So it becomes:

Javascript - to make our websites feel and look dynamic
APIs - our server-side processes and databases live here. They can be self-made or 3rd party
Markup - the structure of our content, our assets and our dependencies, all pre-built and pre-rendered, deployed on a CDN

Rather than targeting specific frameworks, databases and operative systems, like most stack terminologies, the JAMstack ties itself to the core foundations of modern web applications, not tying developers to specific technologies.

The JAMstack is a different philosophy.

Instead of building your pages on a webserver on every request, you would have most of your website pages pre-built and deployed on a CDN. With this, you remove any overhead of maintaining and scaling web servers and or any kind of advanced infrastructure. Instead, you just have static files on a CDN, with super low response times, near-limitless scalability and geographical distribution, so everyone around the world would have a smooth experience while accessing your content.

Behind this core concept, you find the tool that is often associated with the JAMstack, the static site generator. Super in vogue, with most being optimized for instant page load and cutting edge optimizations, static site generators take your code and data sources and they turn it into static files, ready to be deployed to any CDN of your liking.

Most static site generators support all kinds of data sources. Your data may be extracted from any existing service or API that you may have. You can also use CMSs to layout your content, and you can even use... more static files! You can have your data laid out as static markdown files (or any type of file really) and import that on your static site generator.

Then you bundle it all in on a CI/CD pipeline, the only infrastructure you need to manage, that is responsible for rebuilding your website every time you update your code or your data and sending it into a CDN. That's why it's essential to keep track of your code on hosted git services, like Github, as they seamlessly integrate with most CI/CD pipelines to trigger builds. If you are using a CMS, you should also integrate it into this pipeline, so content changes also trigger a new build. Check out the blogpost about CI/CD from our friends at Subvisual if you need more info about the process!

If you feel intimidated by developing your own CI/CD pipeline you can use managed services like Netlify or Now. These platforms integrate directly with hosted git services like Github or Gitlab and they build and deploy your website into a CDN, making the entire process very easy. They also can manage domains, SSL certificates, reducing the entry barrier to production to a few clicks on their dashboards. They offer very generous free plans for developers and hobby projects also, making them a great tool to try out the JAMstack.

Static is not boring

You still can develop highly interactive web applications by resorting more to APIs instead of generated static HTML. If you are not using third-party services, perhaps you still need to maintain some sort of web server and infrastructure to maintain your APIs, but still, the decoupling that you achieve now will allow you to be more productive and you still serve your frontend from a highly capable CDN, which will boost your performance all the same.

But even then, you can write your server-side code without servers, by resorting to serverless platforms, allowing you to deploy server-side code, without requiring any infrastructure whatsoever. Adding to their extensive feature set, Netlify and Now also support deploying serverless functions in Javascript and more languages.

And with modern browsers having a ton of features that were mostly unthinkable in the past, we can push more and more work to the client-side. Speech recognition, 3D graphics, PWAs, Web Assembly and many more tools are available on most browsers. Things that were possible only in native environments are now a reality on the web.

Our stack

To build our website we use a static site generator to build the entire website, and our blog posts are managed by a headless CMS. You can check out the source code in our Github repo. We use Gatsby as a static site generator and Forestry as our CMS. We then build everything using Netlify. With that, we have a wonderful development experience, that allows us to develop features and also fix bugs very quickly.

With near limitless scalability, we don't have to worry about scaling the website, it's just static files on a CDN. Super secure, our third parties all use Github's OAuth and are strictly audited. And finally, the developer experience is just amazing. All it takes is three steps to being able to contribute to our website: clone the git repository, install dependencies, run the bundled development server.

The JAMstack is here to stay, and we are putting it into use whenever we can, to build faster, more accessible and more secure websites. In the end, all that matters is our user's experience.

Resources

You can check these resources and tools to find out more about JAMstack! Check out their blogs for tutorials, guides, and more advanced use cases.

Francisco Sousa's blog

Leveling up your web apps with Zod and ChatGPT

Pick an LLM provider

How to get JSON out of LLMs

Some practical examples

Warning: This code is meant for servers!

UX considerations

Final thoughts

Validating environment variables with zod

Taking it a step further

Handling non-string values

Handling a bad schema

I can’t use process.env!

Using zustand with Svelte

What's a Svelte store?

Zustand

Final thoughts

Typing remix loaders with confidence

Remix, what?

Loaders, what are they?

Let's add types the regular way

Inferring the types from the loader

Final notes

Locale detection with Remix

Why does this happen when using SSR?

How can we fix it then?

Authenticating things with cookies on Next.js

Next.js

Architecture

Disclaimer

Let's get coding

User management

Cookies and JWT

API routes

Frontend

Closing notes

Predictable React authentication with the Context API

Praise be the Context

Some disclaimers

Time to code

Wrapping up

Next.js and Airtable: a tale of empowering no-code tools

Getting technical

Show me the code!

Updating Airtable data

Final notes

Reactive Rails applications with StimulusReflex

So what is this Stimulus Reflex thing?

How the magic happens

The classic todo app

Caveats

Wrapping up

Spicing up your Rails frontend experience

Rails and the modern web

Turbolinks and remote helpers

Components

Stimulus

A fun alternative

Building and hosting a Discord bot

Getting started

The bot

How to put this online

Sinatra

Heroku

Disclaimer

Next Steps

A final note

Let’s slim down on Javascript

Why care?

Ditch SPAs

Use native libraries

Be careful with 3rd party dependencies

Code Splitting

Don't support older browsers

Wrapping up

Dealing with webfont loading

FOIT & FOUT

font-display to the rescue

Smoothing out the FOIT

Wrapping up

I can’t use `process.env`!

Inferring the types from the `loader`

`font-display` to the rescue