Pagination

Navigate through large result sets using cursor-based pagination

Pagination

The OpenMetadata API uses cursor-based pagination for list endpoints. This ensures consistent results even when data changes between requests.

Pagination Parameters

Parameter	Type	Default	Description
limit	integer	10	Number of results per page (max 1000)
before	string	-	Cursor for previous page
after	string	-	Cursor for next page

Response Fields

The paging object contains:

Field	Type	Description
total	integer	Total count of matching resources
before	string	Cursor for the previous page (if available)
after	string	Cursor for the next page (if available)

Examples

Basic Pagination

Basic Pagination

Python

from metadata.sdk import Tables

# Get first page
first_page = Tables.list(limit=20)

print(f"Fetched {len(first_page.entities)} tables")
for table in first_page.entities:
    print(table.fullyQualifiedName)

# Get next page using after cursor
if first_page.after:
    next_page = Tables.list(limit=20, after=first_page.after)
    for table in next_page.entities:
        print(table.fullyQualifiedName)

Iterating Through All Results

Iterating Results

Python

from metadata.sdk import Tables

# Auto-paginate through all tables
for table in Tables.list_all(batch_size=100):
    print(table.fullyQualifiedName)
    # Process each table...

Filtering with Pagination

Combine pagination with filters for efficient data retrieval:

Filtering

Python

from metadata.sdk import Tables

# List tables from a specific database with fields
tables = Tables.list(
    limit=50,
    fields=["owners", "tags"],
    filters={"database": "prod.analytics"},
)

for table in tables.entities:
    print(table.fullyQualifiedName)

Include Fields

Control which fields are returned in the response using the fields parameter:

Include Fields

# Request specific fields
curl -X GET "https://your-company.open-metadata.org/api/v1/tables?fields=owner,tags,columns&limit=20" \
  -H "Authorization: Bearer $TOKEN"

Common field options for tables:

• owner - Include owner information
• tags - Include tags and classifications
• columns - Include column definitions
• followers - Include followers
• tableConstraints - Include constraints
• usageSummary - Include usage statistics

Best Practices

1. Use reasonable page sizes

   Start with limit=50-100. Larger pages reduce API calls but increase memory usage.

2. Don't skip pages

   Always use cursors sequentially. Don’t try to construct cursor values manually.

3. Handle empty results

   Check if data array is empty or after cursor is null to detect end of results.

4. Request only needed fields

   Use the fields parameter to reduce response size and improve performance.

5. Implement retry logic

   Handle transient errors gracefully when paginating through large datasets.

Pagination vs Search

For finding specific resources, consider using the Search API instead of paginating through all results:

Pagination vs Search

# Search is faster for finding specific resources
curl -X GET "https://your-company.open-metadata.org/api/v1/search/query?q=customers&index=table_search_index" \
  -H "Authorization: Bearer $TOKEN"

Search API

Learn about searching and filtering metadata