Need advice on large data export – JournalEntry
We have a monthly process currently in AWS that uses RESTlet calls to NetSuite to run a suitescript that runs a Saved Search to pull journal entry data. It worked fine for years but we have acquired numerous subsidiaries recently and some of them have large amounts of data. The volume last month was over 350,000 rows where it used to be only 25,000. We have been tweaking our integration and breaking up runs into manually executed steps to avoid timeouts – either the 15 minute Lambda timeout or the suitescript timeout. We are using pagination in the suitescript but with 350 pages we still have to send multiple calls to the URL in batches from AWS. We are taking a fresh look at this integration and looking for the best option to get the data from NS to AWS. Any suggestions or examples are appreciated.



