접속 불가능한 MongoDB 샤드 제거 방법

2021. 5. 28. 20:50서버 프로그래밍

아쉽게도 뭔가 놓친부분이 있어서인지 이것만으로는 완전히 복구되지 않았지만, 유용한 팁이니 공유한다.

I tried several options to do this in version 4.2.

At the end I ended to these commands to be executed on Config Server:

use config

db.databases.updateMany( {primary: "shard0002"}, {$set: {primary: "shard0000"} })
db.shards.deleteOne({_id : "shard0002" })
db.chunks.updateMany( {shard : "shard0002"}, {$set: {shard: "shard0000"} })

while ( db.chunks.updateMany( {"history.shard" : "shard0002"}, 
      {$set: {"history.$.shard": "shard0000"} }).modifiedCount > 0 ) { print("Updated") }

It works to a certain extent, i.e. CRUD operations are working. However, when you run getShardDistribution() then you get an error Collection 'db.collection' is not sharded.

Finally I see only one reliable & secure solution:

  • Shut down all mongod and mongos in your sharded cluster
  • Start available shards as standalone service (see Perform Maintenance on Replica Set Members)
  • Take a backup from available shards with mongodump.
  • Drop data folders from all hosts.
  • Build your application newly from scratch. Startup all mongod and mongos
  • Load data into new cluster with mongorestore

Perhaps for large cluster you have to shuffle around a bit like this:

  • Deploy Config servers and mongos server, with one empty shard
    1. Start one old shard as standalone
    2. Take backup from this old Shard
    3. Tear down this old shard
    4. build a fresh empty new shard
    5. add new shard to your new cluster
    6. restore data into the new cluster
    7. backup can be dropped and shard can be reused in new cluster
  • Repeat above for each shard you have in your cluster (the broken shard might be skipped most likely)

https://stackoverflow.com/questions/3861047/remove-inaccessible-mongo-shard/30941779

 

Remove inaccessible Mongo shard

I have a MongoDB sharded setup with 3 shards: shard0000, shard0001 and shard0002. The machine that runs shard0002 is down now, which causes all my queries to fail. I'd like to temporarily remove

stackoverflow.com