본문 바로가기

리눅스

MongoDB의 샤딩을 활성화(enable sharding)하고 샤딩된 클러스터에서 쿼리를 테스트하는 방법

728x90

MongoDB의 샤딩을 활성화(enable sharding)하고 샤딩된 클러스터에서 쿼리를 테스트하는 방법

MongoDB의 샤딩은 데이터베이스를 여러 서버에 분산 저장하고 대량의 데이터를 효율적으로 처리하기 위한 기술입니다.

  • 샤딩 전략
    • 해시 기반 샤딩(Hash-based sharding)
    • 범위 기반 샤딩(Range-based sharding)

 

새 데이터베이스(exampleDB) 생성

use exampleDB
db.stats()

인덱스 생성

db.exampleCollection.createIndex({ name : 1 })
show collections
db.exampleCollection.getShardDistribution()
mongos> db.exampleCollection.getShardDistribution()
Collection exampleDB.exampleCollection is not sharded.
for(var i=1; i<=500000; i++) db.exampleCollection.insert({
"userid" : i,
"name": "user"+i,
"age": Math.floor(Math.random()*100),
"score": Math.floor(Math.random()*100),
"time": new Date()
})

t1

해시 샤딩(hashed sharding)

새 데이터베이스(shardDB) 생성

use shardDB
db
db.stats()

데이터베이스 수준에서 샤딩 활성화(enable sharding)

sh.enableSharding("shardDB")

새 컬렉션(exampleCollection)을 만들고 해당 _id 키를 해시합니다.

db.exampleCollection.createIndex({ "_id" : "hashed" })

컬렉션 샤딩

sh.shardCollection("shardDB.exampleCollection", { _id : "hashed" })

데이터베이스(config) 전환

use config
db.databases.find()
mongos> db.databases.find()
{ "_id" : "shardDB", "primary" : "rs1", "partitioned" : true, "version" : { "uuid" : UUID("3609e37c-743c-46b7-80a9-0fceedcd06a1"), "lastMod" : 1 } }

데이터베이스(shardDB) 전환

use shardDB

샤드 배포 확인

db.exampleCollection.ensureIndex({ _id : "hashed" })
mongos> db.exampleCollection.ensureIndex({ _id : "hashed" })
{
	"raw" : {
		"rs1/shard11:27017,shard12:27017,shard13:27017" : {
			"numIndexesBefore" : 2,
			"numIndexesAfter" : 2,
			"note" : "all indexes already exist",
			"ok" : 1
		},
		"rs2/shard21:27017,shard22:27017,shard23:27017" : {
			"numIndexesBefore" : 2,
			"numIndexesAfter" : 2,
			"note" : "all indexes already exist",
			"ok" : 1
		}
	},
	"ok" : 1,
	"operationTime" : Timestamp(1672669632, 11),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672669634, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

 

for(var i=1; i<=500000; i++) db.exampleCollection.insert({
"userid" : i,
"name": "user"+i,
"age": Math.floor(Math.random()*100),
"score": Math.floor(Math.random()*100),
"time": new Date()
})
mongos> for(var i=1; i<=500000; i++) db.exampleCollection.insert({
... "userid" : i,
... "name": "user"+i,
... "age": Math.floor(Math.random()*100),
... "score": Math.floor(Math.random()*100),
... "time": new Date()
... })

t2

데이터 분포 확인

db.exampleCollection.getShardDistribution()
mongos> db.exampleCollection.getShardDistribution()

Shard rs1 at rs1/shard11:27017,shard12:27017
 data : 278KiB docs : 2883 chunks : 2
 estimated data per chunk : 139KiB
 estimated docs per chunk : 1441

Shard rs2 at rs2/shard21:27017,shard22:27017
 data : 277KiB docs : 2876 chunks : 2
 estimated data per chunk : 138KiB
 estimated docs per chunk : 1438

Totals
 data : 555KiB docs : 5759 chunks : 4
 Shard rs1 contains 50.05% data, 50.06% docs in cluster, avg obj size on shard : 98B
 Shard rs2 contains 49.94% data, 49.93% docs in cluster, avg obj size on shard : 98B
728x90

범위 기반 샤딩(Range-based sharding)

새 데이터베이스(shardDB) 생성

use shardDB
db
db.stats()

데이터베이스 수준에서 샤딩 활성화(enable sharding)

sh.enableSharding("shardDB")
mongos> sh.enableSharding("shardDB")
{
	"ok" : 1,
	"operationTime" : Timestamp(1672672587, 3),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672672587, 3),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

새 컬렉션(exampleCollection)을 만들고 해당 _id 키를 해시합니다.

db.exampleCollection.createIndex({ "_id" : "hashed" })
mongos> db.exampleCollection.createIndex({ "_id" : "hashed" })
{
	"raw" : {
		"rs1/shard11:27017,shard12:27017,shard13:27017" : {
			"createdCollectionAutomatically" : true,
			"numIndexesBefore" : 1,
			"numIndexesAfter" : 2,
			"commitQuorum" : "votingMembers",
			"ok" : 1
		}
	},
	"ok" : 1,
	"operationTime" : Timestamp(1672672596, 3),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672672596, 3),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

컬렉션 샤딩

sh.shardCollection("shardDB.exampleCollection", { _id : "hashed" })
mongos> sh.shardCollection("shardDB.exampleCollection", { _id : "hashed" })
{
	"collectionsharded" : "shardDB.exampleCollection",
	"collectionUUID" : UUID("e2727f13-cc6c-4c95-ba43-9ad42d9017e9"),
	"ok" : 1,
	"operationTime" : Timestamp(1672672607, 13),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672672607, 13),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

데이터베이스(config) 전환

use config
db.databases.find()
mongos> db.databases.find()
{ "_id" : "shardDB", "primary" : "rs1", "partitioned" : true, "version" : { "uuid" : UUID("3aa05940-c903-4646-b89e-b8920055beab"), "lastMod" : 1 } }

데이터베이스(shardDB) 전환

use shardDB

샤드 배포 확인

db.exampleCollection.ensureIndex({ _id : "hashed" })
mongos> db.exampleCollection.ensureIndex({ _id : "hashed" })
{
	"raw" : {
		"rs1/shard11:27017,shard12:27017,shard13:27017" : {
			"numIndexesBefore" : 2,
			"numIndexesAfter" : 2,
			"note" : "all indexes already exist",
			"ok" : 1
		},
		"rs2/shard21:27017,shard22:27017,shard23:27017" : {
			"numIndexesBefore" : 2,
			"numIndexesAfter" : 2,
			"note" : "all indexes already exist",
			"ok" : 1
		}
	},
	"ok" : 1,
	"operationTime" : Timestamp(1672672816, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672672817, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

 

for(var i=1; i<=500000; i++) db.exampleCollection.insert({
"userid" : i,
"name": "user"+i,
"age": Math.floor(Math.random()*100),
"score": Math.floor(Math.random()*100),
"time": new Date()
})
mongos> for(var i=1; i<=500000; i++) db.exampleCollection.insert({
... "userid" : i,
... "name": "user"+i,
... "age": Math.floor(Math.random()*100),
... "score": Math.floor(Math.random()*100),
... "time": new Date()
... })

t3

데이터 분포 확인

db.exampleCollection.getShardDistribution()
mongos> db.exampleCollection.getShardDistribution()

Shard rs1 at rs1/shard11:27017,shard12:27017
 data : 1.39MiB docs : 14639 chunks : 2
 estimated data per chunk : 712KiB
 estimated docs per chunk : 7319

Shard rs2 at rs2/shard21:27017,shard22:27017
 data : 1.36MiB docs : 14360 chunks : 2
 estimated data per chunk : 698KiB
 estimated docs per chunk : 7180

Totals
 data : 2.75MiB docs : 28999 chunks : 4
 Shard rs1 contains 50.47% data, 50.48% docs in cluster, avg obj size on shard : 99B
 Shard rs2 contains 49.52% data, 49.51% docs in cluster, avg obj size on shard : 99B

 

728x90