1# CouchDB Replication Speed Testing Report
2## Introduction
3This report details the replication speed testing for CouchDB by fine-tuning parameters. The primary objective was to measure the speed of document processing in CouchDB by varying the replicator parameters used by couchDB.
4## Methodology
5The speed test was conducted using a script that sends a request to the CouchDB scheduler endpoint to retrieve information about the transferred documents. The script then calculates the speed in terms of documents per second and logs this data into an output CSV file.
6### Setup:
71. Run the `hack/datasync/speedTest.py` file with the relevant parameters.
82. This will clean the CouchDB cluster by running the provided cleanup script and establish port forwarding using `kubectl` and then run the speed test.
9### Parameters:
10The parameters set in the `couchdb-server.yaml` file include:
11- `maxjobs`: Maximum number of jobs.
12- `churn`: Churn value.
13- `interval`: Interval value.
14- `username`: Authentication username.
15- `password`: Authentication password.
16- `serverURL`: Server URL (default is “http://localhost:5984/”).
17- `filterByDatabase`: Specific database ID filter (optional).
18## Results
19The following table showcases the best result from the speed test:
20| Timestamp | Database ID | Docs Written | Speed (docs/sec) | MaxJobs | Churn | Interval |
21|-----------|-------------|--------------|------------------|---------|-------|----------|
22| 2023-08-29 18:11:12.800073 | db---9b387a16540e442a8223e739ba83dbbc---catalog---item | 25570 | 583.89 | 35 | 10 | 1000 |
23| ... | ... | ... | ... | ... | ... | ... |
24
25## Bandwidth Calculation
26While doing preliminary debugging, we found that calculating the bandwidth is helpful to calibrate expectations.
27
28## Conclusion
29Based on the data, the best parameters were MaxJobs:35, Churn: 10, Interval:1000. With a speed of approximately 586 documents per second. A low interval corresponded to the highest increase in the speed, all other things being equal. We found that fine-tuning parameters in CouchDB can significantly impact replication speed as we saw speeds ranging from 261 documents/second to 586 documents/second. The excel datasheet can be found [here](https://ncr.sharepoint.com/:x:/r/sites/NCREdge/_layouts/15/Doc.aspx?sourcedoc=%7BA0159AD1-2748-43A9-96CB-124B8A1AE0FC%7D&file=CouchDB%20FIne%20Tuning.xlsx&action=default&mobileredirect=true)
View as plain text