Skip to main content

Processing Uploads in the Background

Uploaded data shouldn't be sent to Quirrel, but be stored in your own database. Add a new DB entity for it:

model UploadedCSV {
id Number @id @default(autoincrement())
data String

Then when a user uploads something, you insert it into the database and enqueue the resulting record's ID into Quirrel.

// app/mutations/uploadCsvForProcessing
import db from "db"
import csvProcessingQueue from "app/api/process-csv"

export default async function uploadCsvForProcessing(data: string) {
const record = await db.uploadedCsv.create({
data: {data},

await csvProcessingQueue.enqueue(


Our Quirrel Queue then fetches the corresponding data from the database and does the required processing. After that's done, it deletes the database record (alternative: add a flag called "finishedProcessing" and set it to true).

// app/api/process-csv
import db from "db"
import {Queue} from "quirrel/blitz"

export default Queue("api/process-csv", async (uploadId: number) => {
const upload = await db.uploadedCsv.findUnique({
where: {id: uploadId},

await doYourProcessing(

await db.uploadedCsv.delete({where: {id: uploadId}})

Now when you want to know whether an upload has already been processed, you can look it up in your own database:

// app/queries/hasFinishedProcessing
import db from "db"

export default async function hasFinishedProcessing(uploadId: number) {
const count = await db.uploadedCsv.count({where: {uploadId}})
return count === 0