The datastore running aggregation queries is built on top of Elasticsearch (currently version 1.7.x). Elasticsearch provides real-time searches and aggregation capabilities with the Analytics API forming a thin, security-enhanced layer on top of it. Elasticsearch query and aggregation language is used in the examples.
-
The language used in the filter field of the query JSON is described in the Elasticsearch Query DSL documentation.
-
The language used in the aggregations field of the query JSON is fully described in the Elasticsearch Aggregations documentation.
Indexes
Different types of data live in different indexes. For example, job data lives in the jobPart index, and costs data lives in the costs index. Required data can be specified by specifying the index in the aggregation API endpoint URL.
Parent-Child Documents
Parts of the information about a job live in two different documents in the Elasticsearch DB. These documents are in a parent-child relationship. For example, information about a job's project lives in the parent document, while information about the job itself lives in the child document. Depending on the information required, aggregations are either before the data parent-child aggregation or after it but the data aggregation itself should be present in all queries.
Common Data
Analysis
{ "id": string, "createdBy": <User>, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "innerId": number, "name": string, "tags": string[], "trashed": boolean, "trashedBy": <User>, "type": string }
Assignment
{ "name": string, "linguist": <User>, "vendor": <Vendor> }
Submitter Portal
{ "id": string, "createdBy": <User>, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "name": string, "tags": string[], "trashed": boolean, "urlId": string }
Buyer
{ "id": string, "name": string }
Client
{ "id": string, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "externalId": string, "name": string, "tags": string[], "trashed": boolean, }
CostCenter
{ "id": string, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "name": string, "tags": string[], "trashed": boolean, }
Domain
{ "id": string, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "name": string, "tags": string[], "trashed": boolean }
Job
{ "id": string, "createdBy": <User>, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "extension": string, "fileName": string, "groupCount": number, "innerId": "1", "languagePair": string, "lastModified": Date, "localePair": string, "sourceLanguage": string, "sourceLocale": string, "tags": string[], "targetLanguage": string, "targetLocale": string", "taskId": string, "trashed": boolean, "trashedBy": <User>, "uid": string }
JobPart
{ "id": "19", "assignedTo": <Assignment>, "beginIndex": number, "buyer": <Buyer>, "createdBy": <User>, "dateCreated": Date, "dateDue": Date, "endIndex": number, "groupCount": number, "innerId": string, "lastModified": Date, "level": number, "status": string, "tags": [], "uid": string, "workflowStep": <WorkflowStep> }
MtEngine
{ "id": string, "deleted": boolean, "default_: boolean, "includeTags": boolean, "name": string, "tags": string[], "type": string, }
NetRateScheme
{ "id": string, "createdBy": <User>, "dateCreated": Date, "dateDeleted": Date, "default_": boolean, "deleted": boolean, "externalId": string, "name": string, "tags": string[], }
PriceList
{ "id": string, "createdBy": <User>, "currency": string, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "default_": boolean, "deleted": boolean, "name": string, "tags": string[], "trashed": boolean, "unit": string }
Project
{ "id": string, "buyer": <Buyer>, "client": <Client>, "costCenter": <CostCenter>, "createdBy": <User>, "dateCreated": Date, "dateDeleted": Date, "dateDue": Date, "dateTrashed": Date, "deleted": boolean, "domain": <Domain>, "innerId": number, "languagePairs": string[], "localePairs": string[], "mtEngine": <MtEngine>, "name": string, "note": string, "owner": <User>, "sourceLanguage": string, "sourceLocale": string, "status": string, "subDomain": <SubDomain>, "tags": string[], "targetLanguages": string[], "targetLocales": string[], "trashed": boolean, "trashedBy": <User>, "uid": string, "vendor": <Vendor> }
Quote
{ "id": string, "createdBy: <User>, "currency": string, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "innerId": string, "name": string, "status": string, "tags": string[], "trashed": boolean, "trashedBy": <User>, "unit": string }
Service
{ "id": string, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "name": string, "publicName": string, "tags": string[], "trashed": boolean, "type": string }
SubDomain
{ "id": string, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "name": string, "tags": string[], "trashed": boolean }
User
{ "id": number, "active": boolean, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "email": string, "firstName": string, "fullName": string, "innerId": number, "jobTitle": string, "lastName": string, "locale": string, "note": string, "role": string, "tags": string[], "timeZone": string, "trashed": boolean, "userName": string }
Vendor
{ "id": string, "candidate": boolean, "dateCreated": Date, "dateDeleted": Date, "dateTrashed": Date, "deleted": boolean, "tags": string[], "token": string, "trashed": boolean }
WorkflowStep
{ "id": string, "abbreviation": string, "dateDeleted": Date, "dateTrashed": Date, "name": string, "order": number, "tags": string[], "trashed": boolean }
Analysis Index
Data Aggregation
"data": { "children": { "type": "analysisType" } }
Parent Document
{ "automationWidget": <AutomationWidget>, "project": <Project>, "service": <Service> }
Child Document
{ "analysis": <Analysis>, "jobPart": <JobPart>, "job": <Job>, "netRateScheme": <NetRateScheme>, "priority": number, "data": { "mt": { "match0": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match100": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match50": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match75": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match85": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match95": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number } }, "repetitions": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "tm": { "match0": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match100": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match101": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match50": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match75": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match85": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number }, "match95": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number } }, "total": { "characters": number, "pages": number, "percent": number, "segments": number, "words": number } } }
API Index
Data Aggregation
"data": { "children": { "type": "apiType" } }
Parent Document
{ "user": <User> }
Child Document
{ "request": { "date": Date, "host": string, "ipv4": string, "ipv6": string, "location": string, "method": string }, "response": { "date": Date, "duration": number, "status": number }, "api": { "action": string, "asynch": boolean, "type": string, "uri": string, "ver": string } }
Costs Index
Data Aggregation
"data": { "children": { "type": "costsType" } }
Parent Document
{ "automationWidget": <AutomationWidget>, "project": <Project>, "service": <Service> }
Child Document
{ "analysis": <Analysis>, "job": <Job>, "jobPart": <JobPart>, "netRateScheme": <NetRateScheme>, "priceList": <PriceList>, "quote": <Quote>, "workflowStep": <WorkflowStep>, "priority": number, "data": { "mt": { "match0": number, "match100": number, "match50": number, "match75": number, "match85": number, "match95": number }, "repetitions": number, "tm": { "match0": number, "match100": number, "match101": number, "match50": number, "match75": number, "match85": number, "match95": number }, "total": number } }
jobPart Index
Data Aggregation
"data": { "children": { "type": "jobPartType" } }
Parent Document
{ "automationWidget": <AutomationWidget>, "project": <Project>, "service": <Service> }
Child Document
{ "job": <Job>, "jobPart": <JobPart> "data": { "counts": { "chars": { "total": number, "confirmed": number, "notConfirmed": number, "locked": number, "notLocked": number, "confirmedAndLocked": number, "notConfirmedAndLocked": number, "completed": number, "notCompleted": number }, "groups": { "total": number }, "segments": { "total": number, "confirmed": number, "notConfirmed": number, "locked": number, "notLocked": number, "confirmedAndLocked": number, "notConfirmedAndLocked": number, "completed": number, "notCompleted": number, "mt": { "postEdited": number, "relevant": number, "notRelevant": number }, "qa": { "checked": number, "notChecked": number } }, "words": { "total": number, "confirmed": number, "notConfirmed": number, "locked": number, "notLocked": number, "confirmedAndLocked": number, "notConfirmedAndLocked": number, "completed": number, "notCompleted": number }, "qa": { "warnings": number, "ignoredWarnings": number, "notIgnoredWarnings": number } } }
-
HTTP requests are run in Postman.
-
Login through a PM or Admin profile, and obtain a login token using a login API.
-
POST requests to
api/v3/analytics/jobPart?token=<your_login_token>
Total Number of Jobs
Query
{ "aggregations": { "data": { "children": { "type": "jobPartType" } } } }
Response
{ "hits": { "total": 359 }, "aggregations": { "data": { "doc_count": 14417 } } }
Under the aggregations field, the data aggregation responded with 14,417 matching documents shown in the doc_count field.
Total Number of Source Words
Query
{ "aggregations": { "data": { "children": { "type": "jobPartType" }, "aggs": { "wordCount": { "sum": { "field": "data.volume.words" } } } } } }
Response
{ "hits": { "total": 359 }, "aggregations": { "data": { "doc_count": 14417, "wordCount": { "value": 6893067 } } } }
Total Number of Source Words Divided by Target Language
Query
{ "aggregations": { "data": { "children": { "type": "jobPartType" }, "aggs": { "byTargetLanguage": { "terms": { "field": "job.targetLanguage", "size": 3 }, "aggs": { "wordCount": { "sum": { "field": "data.volume.words" } } } } } } } }
Response
{ "hits": { "total": 359 }, "aggregations": { "data": { "byTargetLanguage": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 10071, "buckets": [ { "doc_count": 1835, "wordCount": { "value": 702721 }, "key": "cs" }, { "doc_count": 1491, "wordCount": { "value": 2602529 }, "key": "de" }, { "doc_count": 1020, "wordCount": { "value": 92676 }, "key": "fi" } ] }, "doc_count": 14417 } } }
The buckets entity is shown in the response. When the analytics module is asked to split the data by some category, the result is represented as a list of buckets. Each bucket contains a key that defines what data this bucket represents (in this case a target language) and a value (other aggregations) specific just to this part of the data set.
Total Number of Jobs Divided by Project Status
Query
{ "aggregations": { "projectStatus": { "terms": { "field": "project.status" }, "aggs": { "data": { "children": { "type": "jobPartType" } } } } } }
Reponse
{ "hits": { "total": 359 }, "aggregations": { "projectStatus": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "doc_count": 326, "data": { "doc_count": 14318 }, "key": "NEW" }, { "doc_count": 31, "data": { "doc_count": 89 }, "key": "COMPLETED" }, { "doc_count": 1, "data": { "doc_count": 4 }, "key": "ASSIGNED" }, { "doc_count": 1, "data": { "doc_count": 6 }, "key": "DECLINED_BY_VENDOR" } ] } } }
The Analytics module has been told to split the data by project status before the data aggregation.
Total Number of Jobs From New Projects
Query
{ "filter": { "term": { "project.status": "NEW" } }, "aggregations": { "data": { "children": { "type": "jobPartType" } } } }
Reponse
{ "hits": { "total": 326 }, "aggregations": { "data": { "doc_count": 14318 } } }
The filter field is used in the query. This narrows down the data before following aggregations.