«   2025/02   »
일 월 화 수 목 금 토
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28
Recent Posts
Link
Today
Total
02-26 14:41
300x250
관리 메뉴

변군이글루(Mirror)

MongoDB의 샤딩을 활성화(enable sharding)하고 샤딩된 클러스터에서 쿼리를 테스트하는 방법 본문

리눅스

MongoDB의 샤딩을 활성화(enable sharding)하고 샤딩된 클러스터에서 쿼리를 테스트하는 방법

변군Dev 2023. 1. 2. 23:44
728x90

MongoDB의 샤딩을 활성화(enable sharding)하고 샤딩된 클러스터에서 쿼리를 테스트하는 방법

MongoDB의 샤딩은 데이터베이스를 여러 서버에 분산 저장하고 대량의 데이터를 효율적으로 처리하기 위한 기술입니다.

  • 샤딩 전략
    • 해시 기반 샤딩(Hash-based sharding)
    • 범위 기반 샤딩(Range-based sharding)

 

새 데이터베이스(exampleDB) 생성

use exampleDB
db.stats()

인덱스 생성

db.exampleCollection.createIndex({ name : 1 })
show collections
db.exampleCollection.getShardDistribution()
mongos> db.exampleCollection.getShardDistribution()
Collection exampleDB.exampleCollection is not sharded.
for(var i=1; i<=500000; i++) db.exampleCollection.insert({
"userid" : i,
"name": "user"+i,
"age": Math.floor(Math.random()*100),
"score": Math.floor(Math.random()*100),
"time": new Date()
})

t1

해시 샤딩(hashed sharding)

새 데이터베이스(shardDB) 생성

use shardDB
db
db.stats()

데이터베이스 수준에서 샤딩 활성화(enable sharding)

sh.enableSharding("shardDB")

새 컬렉션(exampleCollection)을 만들고 해당 _id 키를 해시합니다.

db.exampleCollection.createIndex({ "_id" : "hashed" })

컬렉션 샤딩

sh.shardCollection("shardDB.exampleCollection", { _id : "hashed" })

데이터베이스(config) 전환

use config
db.databases.find()
mongos> db.databases.find()
{ "_id" : "shardDB", "primary" : "rs1", "partitioned" : true, "version" : { "uuid" : UUID("3609e37c-743c-46b7-80a9-0fceedcd06a1"), "lastMod" : 1 } }

데이터베이스(shardDB) 전환

use shardDB

샤드 배포 확인

db.exampleCollection.ensureIndex({ _id : "hashed" })
mongos> db.exampleCollection.ensureIndex({ _id : "hashed" })
{
	"raw" : {
		"rs1/shard11:27017,shard12:27017,shard13:27017" : {
			"numIndexesBefore" : 2,
			"numIndexesAfter" : 2,
			"note" : "all indexes already exist",
			"ok" : 1
		},
		"rs2/shard21:27017,shard22:27017,shard23:27017" : {
			"numIndexesBefore" : 2,
			"numIndexesAfter" : 2,
			"note" : "all indexes already exist",
			"ok" : 1
		}
	},
	"ok" : 1,
	"operationTime" : Timestamp(1672669632, 11),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672669634, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

 

for(var i=1; i<=500000; i++) db.exampleCollection.insert({
"userid" : i,
"name": "user"+i,
"age": Math.floor(Math.random()*100),
"score": Math.floor(Math.random()*100),
"time": new Date()
})
mongos> for(var i=1; i<=500000; i++) db.exampleCollection.insert({
... "userid" : i,
... "name": "user"+i,
... "age": Math.floor(Math.random()*100),
... "score": Math.floor(Math.random()*100),
... "time": new Date()
... })

t2

데이터 분포 확인

db.exampleCollection.getShardDistribution()
mongos> db.exampleCollection.getShardDistribution()

Shard rs1 at rs1/shard11:27017,shard12:27017
 data : 278KiB docs : 2883 chunks : 2
 estimated data per chunk : 139KiB
 estimated docs per chunk : 1441

Shard rs2 at rs2/shard21:27017,shard22:27017
 data : 277KiB docs : 2876 chunks : 2
 estimated data per chunk : 138KiB
 estimated docs per chunk : 1438

Totals
 data : 555KiB docs : 5759 chunks : 4
 Shard rs1 contains 50.05% data, 50.06% docs in cluster, avg obj size on shard : 98B
 Shard rs2 contains 49.94% data, 49.93% docs in cluster, avg obj size on shard : 98B
728x90

범위 기반 샤딩(Range-based sharding)

새 데이터베이스(shardDB) 생성

use shardDB
db
db.stats()

데이터베이스 수준에서 샤딩 활성화(enable sharding)

sh.enableSharding("shardDB")
mongos> sh.enableSharding("shardDB")
{
	"ok" : 1,
	"operationTime" : Timestamp(1672672587, 3),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672672587, 3),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

새 컬렉션(exampleCollection)을 만들고 해당 _id 키를 해시합니다.

db.exampleCollection.createIndex({ "_id" : "hashed" })
mongos> db.exampleCollection.createIndex({ "_id" : "hashed" })
{
	"raw" : {
		"rs1/shard11:27017,shard12:27017,shard13:27017" : {
			"createdCollectionAutomatically" : true,
			"numIndexesBefore" : 1,
			"numIndexesAfter" : 2,
			"commitQuorum" : "votingMembers",
			"ok" : 1
		}
	},
	"ok" : 1,
	"operationTime" : Timestamp(1672672596, 3),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672672596, 3),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

컬렉션 샤딩

sh.shardCollection("shardDB.exampleCollection", { _id : "hashed" })
mongos> sh.shardCollection("shardDB.exampleCollection", { _id : "hashed" })
{
	"collectionsharded" : "shardDB.exampleCollection",
	"collectionUUID" : UUID("e2727f13-cc6c-4c95-ba43-9ad42d9017e9"),
	"ok" : 1,
	"operationTime" : Timestamp(1672672607, 13),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672672607, 13),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

데이터베이스(config) 전환

use config
db.databases.find()
mongos> db.databases.find()
{ "_id" : "shardDB", "primary" : "rs1", "partitioned" : true, "version" : { "uuid" : UUID("3aa05940-c903-4646-b89e-b8920055beab"), "lastMod" : 1 } }

데이터베이스(shardDB) 전환

use shardDB

샤드 배포 확인

db.exampleCollection.ensureIndex({ _id : "hashed" })
mongos> db.exampleCollection.ensureIndex({ _id : "hashed" })
{
	"raw" : {
		"rs1/shard11:27017,shard12:27017,shard13:27017" : {
			"numIndexesBefore" : 2,
			"numIndexesAfter" : 2,
			"note" : "all indexes already exist",
			"ok" : 1
		},
		"rs2/shard21:27017,shard22:27017,shard23:27017" : {
			"numIndexesBefore" : 2,
			"numIndexesAfter" : 2,
			"note" : "all indexes already exist",
			"ok" : 1
		}
	},
	"ok" : 1,
	"operationTime" : Timestamp(1672672816, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1672672817, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

 

for(var i=1; i<=500000; i++) db.exampleCollection.insert({
"userid" : i,
"name": "user"+i,
"age": Math.floor(Math.random()*100),
"score": Math.floor(Math.random()*100),
"time": new Date()
})
mongos> for(var i=1; i<=500000; i++) db.exampleCollection.insert({
... "userid" : i,
... "name": "user"+i,
... "age": Math.floor(Math.random()*100),
... "score": Math.floor(Math.random()*100),
... "time": new Date()
... })

t3

데이터 분포 확인

db.exampleCollection.getShardDistribution()
mongos> db.exampleCollection.getShardDistribution()

Shard rs1 at rs1/shard11:27017,shard12:27017
 data : 1.39MiB docs : 14639 chunks : 2
 estimated data per chunk : 712KiB
 estimated docs per chunk : 7319

Shard rs2 at rs2/shard21:27017,shard22:27017
 data : 1.36MiB docs : 14360 chunks : 2
 estimated data per chunk : 698KiB
 estimated docs per chunk : 7180

Totals
 data : 2.75MiB docs : 28999 chunks : 4
 Shard rs1 contains 50.47% data, 50.48% docs in cluster, avg obj size on shard : 99B
 Shard rs2 contains 49.52% data, 49.51% docs in cluster, avg obj size on shard : 99B

 

728x90
저작자표시 비영리 동일조건

'리눅스' 카테고리의 다른 글

[draft] 우분투에 Composer를 설치하고 사용하는 방법  (0) 2023.01.03
growpart 명령어  (0) 2023.01.03
주요 SQL과 NoSQL 데이터베이스 유형 및 특징  (0) 2023.01.02
[draft] MongoDB 5.0+에는 AVX를 지원하는 CPU가 필요함  (0) 2023.01.01
[리눅스] 도커 컨테이너로 몽고디비 클러스터 구성하기(mongodb shard cluster)  (0) 2023.01.01
'리눅스' Related Articles more