redis没有直接使用C语言传统的字符串(以空字符结尾的字符串数组),而是自己构建了一种为简单动态字符串(simple dynamic strring, SDS), 并将SDS作为redis默认的字符串表示。
数据结构
redis定义的SDS由协议头和字符串组成,根据字符串的长度分为五种不同类型的字符串,不同长度的字符串分别使用不用的类型。
header组成
header = len + alloc + flag + buf
header | 释义 |
len | 字符串的长度 |
alloc | 字符串实际占用的存储空间(不包含header和null) |
flag | 字符串类型, sdshdr5 ... sdshdr64 |
buf | 字符串指针 |
SDS类型
类型 | 字符串长度 | 备注 |
sdshdr5 | 0 - 31 | 空字符串使用sdshdr8表示 |
sdshdr8 | 32 - 255 | |
sdshdr16 | 256 - 65535 | |
sdshdr32 | 2^16 - (2^32 - 1 ) | |
sdshdr64 | 2^32 - (2^64 - 1) |
DSD特点
获取字符串的长度复杂度,由C字符串的O(n)优化为O(1)。
DSD 是二进制安全的,可以支持 图片、视频等二进制数据的存储,不会因为字符串中含有空字符而中断。
DSD API并且不会造成缓冲区溢出,dsd系列的api会对字符串动态扩缩容。
空间预分配,减少修改字符串时带来的内存重新分配次数。在修改字符串时,如果修改后的字符串长度小于1MB,那么将会分配两倍字符串长度空间,如果修改后的字符串长度大于1MB,那么将会分配 字符串长度 + 1MB的空间 ,预分配出来的空间,必备后用。
源码解析
1.SDS定义
// __attribute__ ((__packed__)) 用来设置函数属性、类型属性和变量属性,属于gcc的特点 // ##连接两个宏定义,#将宏参数转换为字符串 // 初始化一个sds变量(header) #define SDS_HDR_VAR(T,s) struct sdshdr##T *sh = (void*)((s)-(sizeof(struct sdshdr##T))); // 获取sds的指针(header) #define SDS_HDR(T,s) ((struct sdshdr##T *)((s)-(sizeof(struct sdshdr##T)))) /* Note: sdshdr5 is never used, we just access the flags byte directly. * However is here to document the layout of type 5 SDS strings. */ struct __attribute__ ((__packed__)) sdshdr5 { unsigned char flags; /* 3 lsb of type, and 5 msb of string length */ char buf[]; }; struct __attribute__ ((__packed__)) sdshdr8 { uint8_t len; /* used */ uint8_t alloc; /* excluding the header and null terminator */ unsigned char flags; /* 3 lsb of type, 5 unused bits */ char buf[]; }; struct __attribute__ ((__packed__)) sdshdr16 { uint16_t len; /* used */ uint16_t alloc; /* excluding the header and null terminator */ unsigned char flags; /* 3 lsb of type, 5 unused bits */ char buf[]; }; struct __attribute__ ((__packed__)) sdshdr32 { uint32_t len; /* used */ uint32_t alloc; /* excluding the header and null terminator */ unsigned char flags; /* 3 lsb of type, 5 unused bits */ char buf[]; }; struct __attribute__ ((__packed__)) sdshdr64 { uint64_t len; /* used */ uint64_t alloc; /* excluding the header and null terminator */ unsigned char flags; /* 3 lsb of type, 5 unused bits */ char buf[]; };
2.SDS字符串创建
sds _sdsnewlen(const void *init, size_t initlen, int trymalloc) { void *sh; sds s; char type = sdsReqType(initlen); /* Empty strings are usually created in order to append. Use type 8 * since type 5 is not good at this. */ if (type == SDS_TYPE_5 && initlen == 0) type = SDS_TYPE_8; int hdrlen = sdsHdrSize(type); unsigned char *fp; /* flags pointer. */ size_t usable; assert(initlen + hdrlen + 1 > initlen); /* Catch size_t overflow */ sh = trymalloc? s_trymalloc_usable(hdrlen+initlen+1, &usable) : s_malloc_usable(hdrlen+initlen+1, &usable); if (sh == NULL) return NULL; // 初始化sds结构体 if (init==SDS_NOINIT) init = NULL; else if (!init) memset(sh, 0, hdrlen+initlen+1); // 获取buf的指针 s = (char*)sh+hdrlen; // 获取flag的指针 fp = ((unsigned char*)s)-1; // 获取实际用到的字节数(不包含header和空字符) usable = usable-hdrlen-1; if (usable > sdsTypeMaxSize(type)) usable = sdsTypeMaxSize(type); switch(type) { case SDS_TYPE_5: { *fp = type | (initlen << SDS_TYPE_BITS); break; } case SDS_TYPE_8: { SDS_HDR_VAR(8,s); sh->len = initlen; sh->alloc = usable; *fp = type; break; } case SDS_TYPE_16: { SDS_HDR_VAR(16,s); sh->len = initlen; sh->alloc = usable; *fp = type; break; } case SDS_TYPE_32: { SDS_HDR_VAR(32,s); sh->len = initlen; sh->alloc = usable; *fp = type; break; } case SDS_TYPE_64: { SDS_HDR_VAR(64,s); sh->len = initlen; sh->alloc = usable; *fp = type; break; } } if (initlen && init) memcpy(s, init, initlen); s[initlen] = '\0'; return s; }
3.预分配存储空间
/* Enlarge the free space at the end of the sds string so that the caller * is sure that after calling this function can overwrite up to addlen * bytes after the end of the string, plus one more byte for nul term. * * Note: this does not change the *length* of the sds string as returned * by sdslen(), but only the free buffer space we have. */ sds sdsMakeRoomFor(sds s, size_t addlen) { void *sh, *newsh; size_t avail = sdsavail(s); size_t len, newlen; char type, oldtype = s[-1] & SDS_TYPE_MASK; int hdrlen; size_t usable; /* Return ASAP if there is enough space left. */ if (avail >= addlen) return s; len = sdslen(s); sh = (char*)s-sdsHdrSize(oldtype); newlen = (len+addlen); assert(newlen > len); /* Catch size_t overflow */ if (newlen < SDS_MAX_PREALLOC) newlen *= 2; else newlen += SDS_MAX_PREALLOC; type = sdsReqType(newlen); /* Don't use type 5: the user is appending to the string and type 5 is * not able to remember empty space, so sdsMakeRoomFor() must be called * at every appending operation. */ if (type == SDS_TYPE_5) type = SDS_TYPE_8; hdrlen = sdsHdrSize(type); assert(hdrlen + newlen + 1 > len); /* Catch size_t overflow */ if (oldtype==type) { newsh = s_realloc_usable(sh, hdrlen+newlen+1, &usable); if (newsh == NULL) return NULL; s = (char*)newsh+hdrlen; } else { /* Since the header size changes, need to move the string forward, * and can't use realloc */ newsh = s_malloc_usable(hdrlen+newlen+1, &usable); if (newsh == NULL) return NULL; memcpy((char*)newsh+hdrlen, s, len+1); s_free(sh); s = (char*)newsh+hdrlen; s[-1] = type; sdssetlen(s, len); } usable = usable-hdrlen-1; if (usable > sdsTypeMaxSize(type)) usable = sdsTypeMaxSize(type); sdssetalloc(s, usable); return s; }