Concurrent Requests
The Saner Python SDK lets several threads share one client and make calls through it at the same time. Most of the time an API call spends is waiting for the Saner server to answer, so calls that run side by side finish in roughly the time of the slowest one instead of the sum of all of them.
This is useful wherever one process makes several independent calls: dashboards that gather data from many endpoints, automation that fans out over groups or reports, and services that answer several users from one account.
Note: Sharing works the same way on all three client classes (
SanerClient,PlatformClient,CvemClient). The examples below useSanerClient; on a scoped client, drop the namespace (client.Organization...instead ofclient.platform.Organization...). See the "Client Types" guide for the full comparison.
Basic Example
Create one client and hand its methods to a thread pool:
from concurrent.futures import ThreadPoolExecutor
from saner import SanerClient
with SanerClient(api_key="YOUR_API_KEY", accountid="YOUR_ACCOUNT") as client:
with ThreadPoolExecutor(max_workers=4) as pool:
organizations = pool.submit(client.platform.Organization.get)
accounts = pool.submit(client.platform.Account.get)
configs = pool.submit(client.cvem.Configuration.get)
print(organizations.result())
print(accounts.result())
print(configs.result())The three calls are sent together, and each .result() returns the response to that call alone. Nothing about the methods changes: they are the same ordinary functions you call from a single-threaded script.
Passing Arguments
Pass the method itself to submit(), followed by its arguments. submit() forwards them to the method when a worker thread runs it:
from concurrent.futures import ThreadPoolExecutor
from saner import SanerClient
with SanerClient(api_key="YOUR_API_KEY", accountid="YOUR_ACCOUNT") as client:
with ThreadPoolExecutor(max_workers=4) as pool:
device = pool.submit(client.cvem.Device.getInfo, hostname="device-000001")
org = pool.submit(client.platform.Organization.get, organization="Example")
print(device.result())
print(org.result())Note: Do not call the method inside
submit().pool.submit(client.cvem.Device.getInfo(hostname="device-000001"))runs the call immediately on the current thread and then submits its result, so nothing runs in parallel. Pass the method and its arguments separately, as above.
Running One Method for Many Inputs
The most common pattern is running one method over a list, such as looking up several devices. Keep each future next to the input it came from, so every result and every error can be traced back to its call:
from concurrent.futures import ThreadPoolExecutor
from saner import SanerClient, SanerError
hostnames = ["device-000001", "device-000002", "device-000003", "device-000004"]
with SanerClient(api_key="YOUR_API_KEY", accountid="YOUR_ACCOUNT") as client:
with ThreadPoolExecutor(max_workers=4) as pool:
futures = {
hostname: pool.submit(client.cvem.Device.getInfo, hostname=hostname)
for hostname in hostnames
}
for hostname, future in futures.items():
try:
print(hostname, future.result())
except (SanerError, ValueError) as e:
print(hostname, "failed:", e)A failed call affects only its own future. future.result() raises that call's exception in the thread that reads it: a SanerError subclass for a request that failed, or a ValueError for an argument rejected by parameter validation before anything was sent. The other calls carry on and return normally.
If you only need the results in input order and want the first failure to stop the loop, pool.map() is shorter. Wrap the call in a lambda to pass keyword arguments, and keep the pool inside the client's with block so the client is still open while the calls run:
from concurrent.futures import ThreadPoolExecutor
from saner import SanerClient
hostnames = ["device-000001", "device-000002", "device-000003", "device-000004"]
with SanerClient(api_key="YOUR_API_KEY", accountid="YOUR_ACCOUNT") as client:
with ThreadPoolExecutor(max_workers=4) as pool:
results = list(
pool.map(lambda hostname: client.cvem.Device.getInfo(hostname=hostname), hostnames)
)
for hostname, result in zip(hostnames, results):
print(hostname, result)Seeing the Difference
The script below makes the same six read-only calls twice: first one after another, then from four threads sharing one client. It records when each call starts and finishes, so you can see the calls overlap.
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from saner import SanerClient
def run(name, method, t0):
"""Call one SDK method and record when it started and finished."""
start = time.perf_counter() - t0
result = method()
end = time.perf_counter() - t0
return name, threading.current_thread().name, start, end, result
def report(title, rows, total):
print(f"\n{title}: {total:.2f} s")
for name, thread, start, end, _ in rows:
print(f" {name:37} {thread:10} {start:.2f} s -> {end:.2f} s")
with SanerClient(api_key="YOUR_API_KEY", accountid="YOUR_ACCOUNT") as client:
calls = [
("Organization.get", client.platform.Organization.get),
("Account.get", client.platform.Account.get),
("User.get", client.platform.User.get),
("ServiceProvision.get", client.platform.ServiceProvision.get),
("Configuration.get", client.cvem.Configuration.get),
("Compliance.getAllBenchmarkProvisions", client.cvem.Compliance.getAllBenchmarkProvisions),
]
# One at a time: each call waits for the previous one to finish.
t0 = time.perf_counter()
one_by_one = [run(name, method, t0) for name, method in calls]
one_by_one_total = time.perf_counter() - t0
report("One at a time", one_by_one, one_by_one_total)
# The same client, shared by 4 threads.
t0 = time.perf_counter()
with ThreadPoolExecutor(max_workers=4, thread_name_prefix="worker") as pool:
shared = list(pool.map(lambda call: run(*call, t0), calls))
shared_total = time.perf_counter() - t0
report("Shared by 4 threads", shared, shared_total)
print(f"\n{one_by_one_total / shared_total:.1f}x faster\n")
# Every thread received its own response.
for (name, _, _, _, a), (_, _, _, _, b) in zip(one_by_one, shared):
print(f" {name:37} {'same response' if a == b else 'different'}")Output from one run:
One at a time: 1.09 s
Organization.get MainThread 0.00 s -> 0.25 s
Account.get MainThread 0.25 s -> 0.43 s
User.get MainThread 0.43 s -> 0.58 s
ServiceProvision.get MainThread 0.58 s -> 0.85 s
Configuration.get MainThread 0.85 s -> 0.97 s
Compliance.getAllBenchmarkProvisions MainThread 0.97 s -> 1.09 s
Shared by 4 threads: 0.36 s
Organization.get worker_0 0.00 s -> 0.16 s
Account.get worker_1 0.01 s -> 0.24 s
User.get worker_2 0.01 s -> 0.24 s
ServiceProvision.get worker_3 0.01 s -> 0.31 s
Configuration.get worker_0 0.16 s -> 0.28 s
Compliance.getAllBenchmarkProvisions worker_1 0.24 s -> 0.36 s
3.0x faster
Organization.get same response
Account.get same response
User.get same response
ServiceProvision.get same response
Configuration.get same response
Compliance.getAllBenchmarkProvisions same responseWhat the output shows:
- One at a time (1.09 s). Each call starts only when the previous one ends, so the total is the sum of all six.
- Shared by 4 threads (0.36 s). The first four calls all start within 0.01 s of each other, on four different threads, through the same
client. The last two wait for a free worker:worker_0picks upConfiguration.getthe moment its first call finishes at 0.16 s. - Same response. Every threaded call returned exactly what the same call returned on its own. No thread received another thread's response.
Your timings depend on your server and network. The slower each call is (large reports, long device lists), the more time sharing saves, because the waiting overlaps while the work stays the same.
Choosing the Number of Threads
max_workers is the number of calls in progress at once. A small number such as 4 captures most of the benefit. Keep these limits in mind before raising it:
- Each call occupies its thread until the server answers. Sharing removes queueing between calls, not the cost of a thread per call in progress, up to the
timeoutyou configured. - Each client reuses up to 10 connections per host, separately for RPC and REST calls. More concurrent calls than that still succeed, but each extra connection is opened for one request and then discarded, which costs a new TLS handshake every time.
- More threads means a higher request rate. If the server answers HTTP 429, the SDK retries with backoff (see the "Retry and Timeout Behaviour" guide), but a steady stream of 429s means the worker count is too high.
To work with several accounts at once, create one client per account as shown in the "Multi-Account Usage" guide, and share each of those clients between threads in the same way.
What Each Call Keeps to Itself
Each call keeps its own working state, so calls running together cannot interfere:
- Retries are counted per call. One call retrying a timeout does not use up another call's retries. Retry rules, including the protection against replaying writes, apply to each call exactly as they do without threads.
- Responses and errors belong to the call that made them. An exception raised in a worker thread reaches whoever reads that call's result (
future.result()re-raises it), with its ownstatus_code,method,urlandattempts. - Log records are written whole. With
enable_logging=True, records from concurrent calls never interleave, and each call is logged once. See the "Logging" guide. - No cookies are kept between calls. The client authenticates with your API key on every request, so no call depends on state left behind by another.
Closing a Shared Client
Close a shared client once all of its workers have finished. Nesting the thread pool inside the client's with block, as in the examples above, does this for you: the pool waits for its calls to finish before the client closes.
If one thread does call close() while another thread's call is in progress:
- a request already on the wire still returns its response;
- a call waiting to retry stops and raises
SanerClientClosedErrorinstead of retrying. Itsattemptscounts the requests already sent, which may have reached the server, so check it before re-issuing a write; - a call that starts at the same instant as
close()may either complete or fail with a connection error.
See the "Error Handling Model" guide for the full behaviour of SanerClientClosedError.
What Sharing Does Not Change
- Calls cannot be cancelled once started. Abandoning a
Futurestops you waiting for it, but the request still runs until the server answers or thetimeoutexpires. - The methods stay synchronous. There is no
async/awaitinterface; threads are how the SDK runs calls side by side.
Updated 6 days ago