Finding flaky Cypress tests by simulating a slow network

https://www.loom.com/share/baf4691603084259aa4d2d06e3c517cb

You have some flaky tests in Cypress: sometimes a test fails, sometimes it doesn’t, or different tests fail every time you run your test suite.

They might be flaky because some of your tests are ending while a network request is still pending or a request fires after Cypress thinks the test has ended. Your test tears down, another test sets up, and then your request goes through and fails.

Often you can locate these tests by simulating a delay in your API responses. This is especially helpful if the tests are failing in CI only and not locally. Simulating a delay essentially makes the problem more consistent. It doesn’t just crop up some of the time, when you lose the race condition, it always happens. Here’s an example using GraphQL and Next.js:

// pages/api/graphql.ts

const sleep = (delay: number) => {
  return new Promise<void>((res) => {
    setTimeout(res, delay);
  });
};

export default async function handler(req: NextApolloHybridRequest, res: NextApiResponse) {
	await sleep(500); // simulate slow network

  return server.createHandler({ path: GRAPHQL_PATH })(req, res);
}

And then run your tests again. You'll likely see errors in all the places where this issue happens.

Tip: The culprit is probably the test before the one that fails since it's the one that made the request.

To resolve this, make sure your test confirms that the request went through (using cy.intercept and cy.wait, examples here and here) or that it's looking for something on the page that will only appear when the request is complete (e.g. a success/error message). One caveat with looking for something on the page: your app might be doing optimistic updates (showing that something worked before the request is complete), so it might not be as reliable.

Remember to remove this delay before committing your changes.