News for hackers. Stuff that matters.
Re:Using Elasticsearch as the Primary Data Store in an ETL P... by bobbyi_settv on Friday November 30, 2018 @03:21PM (#18572266)
Yes it does. I'm not sure what you think the refresh operation is if not that.

I've run countless integration tests with ES and never seen something fail due to refresh not working as advertised. If you have, what version of ES was it? Can you give some sample code that sporadically exhibits the problem?

Re:Using Elasticsearch as the Primary Data Store in an ETL P... by kn7 on Friday November 30, 2018 @03:44PM (#18572467)
We generally use index refresh in ITs (running ES in Docker) and it fails occasionally, which I believe the case described here: "The (near) real-time capabilities depend on the index engine used." https://www.elastic.co/guide/en/elasticsearch/reference/curr...
Re:Using Elasticsearch as the Primary Data Store in an ETL P... by manigandham on Friday November 30, 2018 @04:34PM (#18572858)
That seems to be the issue then. The refresh flag should be passed in your insert/update/delete operations.

The refresh command can also be called (which is what you're doing) but this is a different operation and just triggers the index build with no guarantees that it finishes or is consistent with any particular data mutation.

Did you read the previously posted documentation for the refresh flag?