Pagination
Long lists come one page at a time. Live lists use page numbers and history uses cursors.
page starts at 1. limit is how many entries you get per page, up to a maximum that depends on the endpoint. Leave both out and you get the first page at the default size.
curl "https://api.donutstats.io/v1/leaderboards/money?page=3&limit=500" \
-H "x-api-key: $DONUTSTATS_API_KEY"Paginated lists share one shape, the Page object. total counts every entry that matches your filters, so the number of pages is Math.ceil(total / limit).
{
"page": 1,
"limit": 2,
"total": 14033,
"updatedAt": 1792724121936,
"entries": [
"one Order per entry"
]
}Keep asking for the next page until you have seen total entries:
async function* everyPage(path, limit = 200) {
for (let page = 1; ; page++) {
const separator = path.includes("?") ? "&" : "?";
const response = await fetch(
`https://api.donutstats.io${path}${separator}page=${page}&limit=${limit}`,
{ headers: { "x-api-key": process.env.DONUTSTATS_API_KEY } },
);
if (!response.ok) throw new Error(`DonutStats API returned ${response.status}`);
const body = await response.json();
yield* body.entries;
if (page * limit >= body.total) return;
}
}
for await (const order of everyPage("/v1/orders?item=diamond")) {
console.log(order.unitPrice, order.remaining);
}Markets change while you page. A listing can sell between your requests and shift everything after it up by one, so the same entry can appear twice or be skipped. Two habits make that harmless:
- Deduplicate by
id. Listing and order ids never change. - Filter instead of reading everything. An item or category filter is faster and smaller.
Leaderboards barely move
Every leaderboard page comes from the latest complete snapshot, never a partial one. A new snapshot only takes over once it is finished, every 6 hours, so reading the whole baltop in one go almost always sees a single ranking. Compare board.updatedAt between pages if you need to be sure.
History endpoints can hold millions of entries, so they page with a cursor instead of page numbers. Every response is a Feed: entries, the limit you asked for, and next. Pass next back as cursor to get the following page. When next is null you have everything.
{
"limit": 2,
"next": "MTc5MjcyNzYzMzAxOC4yMjAxNzk1OA",
"entries": [
"one ClosedListing per entry"
]
}Cursors have three useful properties:
- Deep pages are as fast as the first one, however far back you go.
- Nothing is skipped or repeated when new entries arrive while you page, because a cursor points at a position in time rather than a row number.
- They combine with
fromandto, in Unix milliseconds, to read one window of time. Keep the same filters on every page.
Treat cursors as opaque
A cursor is a short string. Store it if you want to resume later, but do not build or change one yourself. A malformed cursor is a 400.
Follow next until it runs out. With the biggest page size most history fits in a handful of requests:
async function* everyEntry(path) {
let cursor = null;
do {
const url = new URL(`https://api.donutstats.io${path}`);
if (cursor) url.searchParams.set("cursor", cursor);
const response = await fetch(url, {
headers: { "x-api-key": process.env.DONUTSTATS_API_KEY },
});
if (!response.ok) throw new Error(`DonutStats API returned ${response.status}`);
const body = await response.json();
yield* body.entries;
cursor = body.next;
} while (cursor);
}
for await (const fill of everyEntry("/v1/orders/fills?item=bone&limit=1000")) {
console.log(fill.at, fill.quantity, fill.paid);
}Page numbers go up to 100,000. Cursors have no limit.