You are trying to create a simple Job Info (EmpJob entity) integration, but the time-based filter is not working as expected and returning inaccurate records. Specifically, the 'Modified Since' filter is not giving the desired results.
- When you use the the 'Modified Since' filter in combination with the 'Effective From/To' filter to include all records, you do not get the desired results. This grabs ALL records of a given employee where at least one record has been modified since last run time, and thus does not limit it to only the record(s) recently modified.
- If you remove the 'Effective From/To' filter, this grabs the current record only (even if historical records are recently modified [they fall out of the results], and even if the current record was not recently modified).
Your desired results is to get all Job Info records (current and/or historical) that have been last modified since the last time the integration ran.
You have not been able to configure the integration filters to accomplish this requirement.
SuccessFactors Integration Center
SuccessFactors OData API
Reproducing the Issue
- Navigate to Admin Center > Integration Center > My Integrations
- Click to Edit the Integration
- Investigate the filter options selected In the preview (Configure Fields tab)
The basic idea always is that the lastModified query checks if an object was changed, not if a "time slice" was changed.
- Implement one interface that send all the EmpJob data every execution ("Effective From" = 1900-01-01 and "Effective To" = 9999-12-31).
- Store the values received in some internal staging table.
- Build an additional logic in your other system, doing a comparison between the internal staging table with the values received from the last run, calculating the "Delta" to see the changes from the last run.
- Build one OData interfaces using the filters "Effective From" = 1900-01-01 and "Effective To" = 9999-12-31 (fromDate=1900-01-01&toDate=9999-12-31) to retrieve all the registers from the entity.
- After receiving the OData Response, build an additional logic in the middleware like Dell Boomi or HCI to filter out only the registers with "Last modified Date Time" greater than the last run.
- This logic can be implemented in the middlewares, but not in the Integration Center.
The official document about this behavior is https://help.sap.com/viewer/b2b06831c2cb4d5facd1dfde49a7aab5/latest/en-US/73cfd57b6db440adb7b8a6f5d5ac018a.html
- Effective To
- lastModifiedDateTime ge datetimeoffset
- Modified Since
- Last modified Date Time
- Effective Date
- Time based filters
- Integration Center