I believe there is no answer to optimized "lfs find" but still trying to see if
I can learn more if there is.
Q1) I have been trying to scan 39 OST's using lfs find and this is taking forever? Are
there any tips or tricks to speed this up. Scans are taking anywhere between 15-24 hours
per OST's to finish if all goes well without interruption. I am parallelizing my
scan's from multiple clients to speed this up but don't know of any alternate
Q2) On the other hand when I start "lfs find" on each of the 39 OST's I have
doomed my MDS server with kernel panic due to out of memory issue. Any tips on how can I
minimize this load and avoid MDS from running out of memory?
Q3) Situation: When a client dies for any reason or if the "lfs find" command
that it is running times out, with Input/Out error "or" transport shutdown(Never
saw this until I started running multiple lfs find and simultaneously running lfs_migrate
for the files that were identified to be moved out of OST's)
Observation: I believe the MDS continues to run and serve the scan request for "lfs
find on a thread until it fails or notices that it has evicted the client, hence taking up
the resources on MDS. I don't know if this makes sense but I am guessing this is
causing my MDS to be loaded with RPC requests and causing further slowdown. Any tunable
I wish there was some kind of indexing we could do to avoid deep scans.