NoSQL#

NoSQL can be understood as “Not Only SQL” or “No SQL.” Compared with traditional relational databases, it is a special kind of database with major differences, so it is also called a non-relational database.

• Comparison with relational databases

Traditional relational databases store structured data. Each table has strict constraints such as field names, field types, and field constraints, and inserted data must follow these rules.

• NoSQL databases usually do not strictly constrain data formats. They are often more flexible and can be key-value, document, or graph based.

Tables in traditional databases are often related to each other, for example through foreign keys.

Non-relational databases generally do not have built-in table relationships. Relationships must be maintained either by business logic in code or by coupling between pieces of data.

Traditional relational databases use SQL for queries, with a relatively unified syntax standard.

By contrast, different NoSQL databases can have very different query syntaxes.

Traditional relational databases can satisfy ACID transaction properties.

NoSQL databases often do not support transactions, or cannot strictly guarantee ACID properties, and instead provide basic consistency.

• Storage method
  • Relational databases are disk-based, which involves a lot of disk I/O and can affect performance.
  • NoSQL databases rely more on in-memory operations. Memory read/write speed is much faster, so performance is usually better.
• Scalability
  • Relational database clustering is usually master-slave, where data consistency between master and slave is used for backup; this is often considered vertical scaling.
  • NoSQL databases can split data across multiple machines, store massive amounts of data, and solve memory size limitations. This is horizontal scaling.
  • Because relational databases have inter-table relationships, horizontal scaling can make data queries much more complicated.

Redis#

Redis stands for Remote Dictionary Server. It is an in-memory key-value NoSQL database.

Features:

• Key-value model; values support many data structures and are very flexible
• Single-threaded, with each command being atomic
• Low latency and high speed (thanks to memory, I/O multiplexing, and efficient implementation)
• Supports data persistence
• Supports master-slave replication and sharding clusters
• Supports multi-language clients

Redis Installation#

Install via Docker

1
docker search redis
2
docker pul redis
3
docker run --restart=always -p 6379:6379 --name myredis -v /home/redis/myredis/myredis.conf:/etc/redis/redis.conf -v /home/redis/myredis/data:/data -d redis redis-server /etc/redis/redis.conf  --appendonly yes
4

5
docker exec -it <容器名> /bin/bash

Redis Data Structure Overview#

Redis is a key-value database. Keys are usually of type String, while values can be of many different types:

Redis Generic Commands#

Generic commands can be used across multiple data types. Common ones include:

• KEYS: View all keys matching a pattern

1
127.0.0.1:6379> keys *
2

3
# 查询以a开头的key
4
127.0.0.1:6379> keys a*
5
1) "age"

In production environments, KEYS is not recommended because it becomes inefficient when there are too many keys.

• DEL: Delete a specified key

1
127.0.0.1:6379> del name #删除单个
2
(integer) 1  #成功删除1个
3

4
127.0.0.1:6379> keys *
5
1) "age"
6

7
127.0.0.1:6379> MSET k1 v1 k2 v2 k3 v3 #批量添加数据
8
OK
9

10
127.0.0.1:6379> keys *
11
1) "k3"
12
2) "k2"
13
3) "k1"
14
4) "age"
15

16
127.0.0.1:6379> del k1 k2 k3 k4
17
(integer) 3   #此处返回的是成功删除的key，由于redis中只有k1,k2,k3 所以只成功删除3个，最终返回
18

19
127.0.0.1:6379> keys * #再查询全部的key
20
1) "age"  #只剩下一个了

• EXISTS: Check whether a key exists

1
127.0.0.1:6379> exists age
2
(integer) 1
3

4
127.0.0.1:6379> exists name
5
(integer) 0

• EXPIRE: Set a TTL for a key; the key is automatically deleted when it expires
• TTL: Check the remaining lifetime of a key

1
127.0.0.1:6379> expire age 10
2
(integer) 1
3

4
127.0.0.1:6379> ttl age
5
(integer) 8
6

7
127.0.0.1:6379> ttl age
8
(integer) -2
9

10
127.0.0.1:6379> ttl age
11
(integer) -2  #当这个key过期了，那么此时查询出来就是-2
12

13
127.0.0.1:6379> keys *
14
(empty list or set)
15

16
127.0.0.1:6379> set age 10 #如果没有设置过期时间
17
OK
18

19
127.0.0.1:6379> ttl age
20
(integer) -1  # ttl的返回值就是-1

Redis Commands - String#

The String type is the simplest storage type in Redis.

Its value is a string, but based on the format it can be divided into three categories:

• string: regular string
• int: integer type, supports increment/decrement
• float: floating-point type, supports increment/decrement

Common String commands include:

• SET: Add or modify a String key-value pair
• GET: Get the String value by key
• MSET: Batch set multiple String key-value pairs
• MGET: Batch get String values by multiple keys
• INCR: Increment an integer key by 1
• INCRBY: Increment an integer key by a specified step, e.g. incrby num 2
• INCRBYFLOAT: Increment a floating-point value by a specified step
• SETNX: Set a String key-value pair only if the key does not exist
• SETEX: Set a String key-value pair with an expiration time

Redis Commands - Hierarchical Key Naming#

Redis does not have a concept like MySQL tables, so how do we distinguish different types of keys?

Redis keys can use multiple words to form a hierarchical structure, separated by :.

This format is not fixed; you can add or remove segments based on your own needs.

For example, if the project name is heima and we have user and product data, we can define keys like this:

• User-related key: heima:user<1>
• Product-related key: heima:product<1>

If the value is a Java object (for example, a User object), you can serialize it into a JSON string and store it:

Once data is stored in Redis this way, visual tools can display keys hierarchically, which makes data management more convenient.

Redis Commands - Hash#

The Hash type is an unordered dictionary, similar to a HashMap in Java.

With the String approach, objects are stored as serialized JSON strings, which is inconvenient when you need to modify a single field.

The Hash structure stores each field of an object independently, making CRUD operations on individual fields easier.

Common Hash commands

• HSET key field value: Add or modify the value of a field in a hash key
• HGET key field: Get the value of a field in a hash key
• HMSET: Batch set multiple fields in a hash key
• HMGET: Batch get multiple fields from a hash key
• HGETALL: Get all fields and values in a hash key
• HKEYS: Get all fields in a hash key
• HINCRBY: Increment a numeric field in a hash key by a specified step
• HSETNX: Set a field in a hash key only if that field does not exist

Redis Commands - List#

The List type in Redis is similar to Java’s LinkedList. It can be viewed as a doubly linked list and supports both forward and reverse access.

Its characteristics are also similar to LinkedList:

• Ordered
• Elements can be duplicated
• Fast insertion and deletion
• Average query performance

It is commonly used to store ordered data, such as like lists or comment lists.

Common List commands include:

• LPUSH key element ...: Insert one or more elements on the left side of the list
• LPOP key: Remove and return the first element on the left side of the list; returns nil if none exists
• RPUSH key element ...: Insert one or more elements on the right side of the list
• RPOP key: Remove and return the first element on the right side of the list
• LRANGE key start end: Return all elements in an index range
• BLPOP / BRPOP: Similar to LPOP and RPOP, but wait for a specified time when no element exists instead of returning nil immediately

Redis Commands - Set#

Redis Set is similar to Java’s HashSet, and can be viewed as a HashMap whose values are null. Since it is also hash-table-based, it has characteristics similar to HashSet:

• Unordered
• Elements are unique
• Fast lookup
• Supports intersection, union, and difference operations

Common Set commands

• SADD key member ...: Add one or more elements to a set
• SREM key member ...: Remove specified elements from a set
• SCARD key: Return the number of elements in a set
• SISMEMBER key member: Check whether an element exists in a set
• SMEMBERS: Get all elements in a set
• SINTER key1 key2 ...: Compute the intersection of sets
• SDIFF key1 key2 ...: Compute the difference of sets
• SUNION key1 key2 ...: Compute the union of sets

Redis Commands - SortedSet#

Redis SortedSet is a sortable set collection, somewhat similar to Java’s TreeSet, but with a very different underlying data structure. Each element in a SortedSet has a score, and elements are ordered by that score. The underlying implementation combines a skip list (SkipList) and a hash table.

SortedSet has the following characteristics:

• Sortable
• Unique elements
• Fast lookup

Because of its sortable nature, SortedSet is often used to implement leaderboard-like features.

Common SortedSet commands include:

• ZADD key score member: Add one or more elements to a sorted set; if an element exists, update its score
• ZREM key member: Remove a specified element from a sorted set
• ZSCORE key member: Get the score of a specified element in a sorted set
• ZRANK key member: Get the ascending rank of a specified element in a sorted set
• ZCARD key: Get the number of elements in a sorted set
• ZCOUNT key min max: Count elements whose score is within a given range
• ZINCRBY key increment member: Increment the score of a specified element by a given amount
• ZRANGE key min max: Get elements within a specified rank range after sorting by score
• ZRANGEBYSCORE key min max: Get elements within a specified score range after sorting by score
• ZDIFF, ZINTER, ZUNION: Difference / intersection / union operations

Note: All ranking commands are ascending by default. For descending order, add REV (or use the reverse variants), for example:

• Ascending rank of a specified element in a sorted set: ZRANK key member
• Descending rank of a specified element in a sorted set: ZREVRANK key member

Jedis Basics#

Dependency:

1
<!--jedis-->
2
<dependency>
3
    <groupId>redis.clients</groupId>
4
    <artifactId>jedis</artifactId>
5
    <version>3.7.0</version>
6
</dependency>

Test:

1
private Jedis jedis;
2

3
@BeforeEach
4
void setup() {
5
    // 1.建立连接
6
    // jedis = new Jedis("192.168.150.101", 6379);
7
//        jedis = JedisConnectionFactory.getJedis();
8
    jedis = new Jedis("127.0.0.1",6379);
9
    // 2.设置密码
10
//        jedis.auth("123321");
11
    // 3.选择库
12
    jedis.select(0);
13
}
14

15
@Test
16
void testString() {
17
    // 存入数据
18
    String result = jedis.set("name", "虎哥");
19
    System.out.println("result = " + result);
20
    // 获取数据
21
    String name = jedis.get("name");
22
    System.out.println("name = " + name);
23
}
24

25
@Test
26
void testHash() {
27
    // 插入hash数据
28
    jedis.hset("user:1", "name", "Jack");
29
    jedis.hset("user:1", "age", "21");
30

31
    // 获取
32
    Map<String, String> map = jedis.hgetAll("user:1");
33
    System.out.println(map);
34
}
35

36
@AfterEach
37
void tearDown() {
38
    if (jedis != null) {
39
        jedis.close();
40
    }
41
}

Jedis Connection Pool#

Jedis itself is not thread-safe, and frequently creating and destroying connections causes performance overhead. Therefore, it is recommended to use a Jedis connection pool instead of direct Jedis connections.

Pooling is not only used here; it is a common design idea in many places, such as database connection pools and thread pools in Tomcat.

1
public class JedisConnectionFacotry {
2

3
     private static final JedisPool jedisPool;
4

5
     static {
6
         //配置连接池
7
         JedisPoolConfig poolConfig = new JedisPoolConfig();
8
         poolConfig.setMaxTotal(8);
9
         poolConfig.setMaxIdle(8);
10
         poolConfig.setMinIdle(0);
11
         poolConfig.setMaxWaitMillis(1000);
12
         //创建连接池对象
13
         jedisPool = new JedisPool(poolConfig,"127.0.0.1",6379,1000);
14
     }
15

16
     public static Jedis getJedis(){
17
          return jedisPool.getResource();
18
     }
19
}

Get a Jedis connection from JedisFactory:

1
@BeforeEach
2
void setup() {
3
    // 建立连接
4
    jedis = JedisConnectionFacotry.getJedis();
5
    // 选择库
6
    jedis.select(0);
7
}

SpringDataRedis Basics#

pom dependencies:

1
<!--redis依赖-->
2
<dependency>
3
    <groupId>org.springframework.boot</groupId>
4
    <artifactId>spring-boot-starter-data-redis</artifactId>
5
</dependency>
6
<!--common-pool-->
7
<dependency>
8
    <groupId>org.apache.commons</groupId>
9
    <artifactId>commons-pool2</artifactId>
10
</dependency>
11
<!--Jackson依赖-->
12
<dependency>
13
    <groupId>com.fasterxml.jackson.core</groupId>
14
    <artifactId>jackson-databind</artifactId>
15
</dependency>
16
<dependency>
17
    <groupId>org.projectlombok</groupId>
18
    <artifactId>lombok</artifactId>
19
    <optional>true</optional>
20
</dependency>

YAML configuration:

1
spring:
2
  redis:
3
    host: 127.0.0.1
4
    port: 6379
5
#    password: 123321
6
    lettuce:
7
      pool:
8
        max-active: 8  #最大连接
9
        max-idle: 8   #最大空闲连接
10
        min-idle: 0   #最小空闲连接
11
        max-wait: 100ms #连接等待时间

Test:

1
@SpringBootTest
2
class JedisDemoApplicationTests {
3

4
    @Autowired
5
    private RedisTemplate redisTemplate;
6

7
    @Test
8
    void testString(){
9
        redisTemplate.opsForValue().set("name","hg");
10

11
        Object name = redisTemplate.opsForValue().get("name");
12
        System.out.println(name);
13
    }
14

15
}

• Add the spring-boot-starter-data-redis dependency
• Configure Redis connection information in application.yml
• Inject RedisTemplate

Data Serialization#

RedisTemplate can accept any Object as a value and write it to Redis:

Before writing, the object is serialized into bytes. By default, JDK serialization is used, and the result looks like this:

Disadvantages:

• Poor readability
• Higher memory usage

Custom serialization strategy

1
@Configuration
2
public class RedisConfig {
3

4
    @Bean
5
    public RedisTemplate redisTemplate(RedisConnectionFactory connectionFactory){
6
        // 创建RedisTemplate对象
7
        RedisTemplate template = new RedisTemplate();
8
        // 设置连接工厂
9
        template.setConnectionFactory(connectionFactory);
10
        // 创建JSON序列化工具
11
        GenericJackson2JsonRedisSerializer jsonRedisSerializer =
12
                new GenericJackson2JsonRedisSerializer();
13
        // 设置Key的序列化
14
        template.setKeySerializer(RedisSerializer.string());
15
        template.setHashKeySerializer(RedisSerializer.string());
16
        // 设置Value的序列化
17
        template.setValueSerializer(jsonRedisSerializer);
18
        template.setHashValueSerializer(jsonRedisSerializer);
19
        // 返回
20
        return template;
21
    }
22

23
}

StringRedisTemplate#

To know the object type during deserialization, the JSON serializer writes the class type into the JSON stored in Redis, which introduces extra memory overhead.

To reduce memory usage, we can use manual serialization. In other words, instead of relying on the default serializer, we control the serialization process ourselves and only use the String serializer. This avoids storing extra type metadata in Redis values and saves memory.

1
@Autowired
2
private StringRedisTemplate stringRedisTemplate;
3

4
private static final ObjectMapper mapper = new ObjectMapper();
5

6
@Test
7
void testSaveUser() throws JsonProcessingException {
8
    // 创建对象
9
    User user = new User("hg", 21);
10
    // 手动序列化
11
    String json = mapper.writeValueAsString(user);
12
    // 写入数据
13
    stringRedisTemplate.opsForValue().set("user:200", json);
14

15
    // 获取数据
16
    String jsonUser = stringRedisTemplate.opsForValue().get("user:200");
17
    // 手动反序列化
18
    User user1 = mapper.readValue(jsonUser, User.class);
19
    System.out.println("user1 = " + user1);
20
}

Two practical serialization approaches for RedisTemplate:

• Option 1:
  • Customize RedisTemplate
  • Change the RedisTemplate serializer to GenericJackson2JsonRedisSerializer
• Option 2:
  • Use StringRedisTemplate
  • Manually serialize objects to JSON before writing to Redis
  • Manually deserialize JSON back into objects after reading from Redis