Avatar

upgrading from 5.6 to 5.7: suggestion for feeds and resultsLimit

0

There will be times when I have a versionNumber from the last time we grabbed a feed (say, StatusData), but getting the feed by passing the fromVersion without a resultsLimit will return too much data. So I plan on getting the feed with a resultsLimit of some reasonable amount of records, and keep calling until I'm "caught up". And that is where I'm possibly going to get myself into trouble.

See, in the past,  I would have a min and max entityid, so I would execute the calls in chunks of x records. But in 5.7's feeds, I could keep going indefinitely if I have to depend on the count being 0 on a call to the feed.

Do you have any suggestions? 

Mark Faucher

Please sign in to leave a comment.

1 comment

Avatar

Hey Mark,

Yup, you approach sounds good. I think the default limit (if you don't supply) is 50,000 records but you can definitely limit it chunks smaller than that as well.

If I understand correctly you are worried that you could come to a situation when there is so much data is coming into that there is never a point where the results count is 0. In theory, yes this could happen, and you could cater for this situation with some code to protect against an endless loop. Something like, if the date of the first record is later than the time (UTC) you started the loop, quit. That being said, it is unlikely to see this happen in a real world situation.

Steve

0
Comment actions Permalink
Steve Hansen 0 votes