Need advice on large data export – JournalEntry

We have a monthly process currently in AWS that uses RESTlet calls to NetSuite to run a suitescript that runs a Saved Search to pull journal entry data.  It worked fine for years but we have acquired numerous subsidiaries recently and some of them have large amounts of data.  The volume last month was over 350,000 rows where it used to be only 25,000.  We have been tweaking our integration and breaking up runs into manually executed steps to avoid timeouts – either the 15 minute Lambda timeout or the suitescript timeout.  We are using pagination in the suitescript but with 350 pages we still have to send multiple calls to the URL in batches from AWS.  We are taking a fresh look at this integration and looking for the best option to get the data from NS to AWS.  Any suggestions or examples are appreciated.

Rookie Asked on May 1, 2024 in Saved Searches.
Add Comment
0 Answer(s)

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.
  • This site made possible by our sponsors:   Tipalti   Celigo   SuiteMigration   Become a Sponsor