构建集团数字档案馆系统：从零到一的全栈实战指南
一、核心架构设计与技术选型
集团数字档案馆系统需要处理海量异构数据，必须采用微服务架构。核心服务包括：档案采集服务、元数据管理服务、全文检索服务、存储管理服务和权限控制服务。
1.1 基础技术栈
后端使用Spring Boot 3.1.5 + Spring Cloud 2022.0.4，数据库采用PostgreSQL 15.3用于结构化数据存储，MongoDB 6.0用于非结构化数据存储，Elasticsearch 8.11.1用于全文检索。
1.2 存储策略设计
根据档案价值等级设计三级存储策略：
- 热存储：SSD存储，存放最近3年高频访问档案
- 温存储：HDD存储，存放3-10年档案
- 冷存储：对象存储（如MinIO），存放10年以上档案
二、环境搭建与依赖配置
2.1 基础环境安装
在CentOS 8服务器上执行以下命令：
``` 安装JDK 17 sudo yum install -y java-17-openjdk-devel java -version 安装Docker sudo yum install -y yum-utils sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo sudo yum install -y docker-ce docker-ce-cli containerd.io sudo systemctl start docker sudo systemctl enable docker 安装Docker Compose sudo curl -L "https://github.com/docker/compose/releases/download/v2.20.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose ```2.2 数据库服务部署
创建docker-compose.yml文件：
``` version: '3.8' services: postgres: image: postgres:15.3 container_name: archive-postgres environment: POSTGRES_DB: digital_archive POSTGRES_USER: archive_admin POSTGRES_PASSWORD: YourStrongPassword123! ports: - "5432:5432" volumes: - postgres_data:/var/lib/postgresql/data networks: - archive-network mongodb: image: mongo:6.0 container_name: archive-mongodb environment: MONGO_INITDB_ROOT_USERNAME: root MONGO_INITDB_ROOT_PASSWORD: YourMongoPassword123! ports: - "27017:27017" volumes: - mongo_data:/data/db networks: - archive-network elasticsearch: image: elasticsearch:8.11.1 container_name: archive-es environment: - discovery.type=single-node - xpack.security.enabled=false - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ports: - "9200:9200" volumes: - es_data:/usr/share/elasticsearch/data networks: - archive-network minio: image: minio/minio:latest container_name: archive-minio environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: minioadmin123 ports: - "9000:9000" - "9001:9001" volumes: - minio_data:/data command: server /data --console-address ":9001" networks: - archive-network volumes: postgres_data: mongo_data: es_data: minio_data: networks: archive-network: driver: bridge ```执行docker-compose up -d启动所有服务。
三、核心服务开发实现
3.1 项目结构初始化
使用Spring Initializr创建父项目：
``` 创建父项目 mvn archetype:generate -DgroupId=com.group.archive \ -DartifactId=digital-archive-parent \ -DarchetypeArtifactId=maven-archetype-quickstart \ -DinteractiveMode=false 创建子模块 cd digital-archive-parent mvn archetype:generate -DgroupId=com.group.archive \ -DartifactId=archive-collector \ -DarchetypeArtifactId=maven-archetype-quickstart \ -DinteractiveMode=false ```重复创建其他子模块：archive-metadata、archive-search、archive-storage、archive-auth。
3.2 档案采集服务实现
在archive-collector模块中添加依赖：
```创建文件解析服务FileParserService.java：
``` @Service public class FileParserService { private final Tika tika = new Tika(); public ArchiveDocument parseFile(MultipartFile file) throws IOException { ArchiveDocument document = new ArchiveDocument(); // 提取基础信息 document.setFileName(file.getOriginalFilename()); document.setFileSize(file.getSize()); document.setMimeType(tika.detect(file.getInputStream())); // 提取文本内容 String content = tika.parseToString(file.getInputStream()); document.setContent(content); // 提取元数据 Metadata metadata = new Metadata(); try (InputStream is = file.getInputStream()) { tika.parse(is, metadata); } Map3.3 元数据管理服务配置
创建元数据实体和Repository：
``` @Entity @Table(name = "archive_metadata") public class ArchiveMetadata { @Id @GeneratedValue(strategy = GenerationType.UUID) private String id; @Column(nullable = false) private String documentId; @Column(nullable = false) private String metaKey; @Column(columnDefinition = "TEXT") private String metaValue; @Column(nullable = false) private String dataType; // STRING, NUMBER, DATE, BOOLEAN @Column(nullable = false) private Boolean isIndexed = false; @Column(nullable = false) @CreationTimestamp private LocalDateTime createdAt; } @Repository public interface MetadataRepository extends JpaRepository3.4 全文检索服务集成
配置Elasticsearch客户端：
``` @Configuration public class ElasticsearchConfig { @Value("${elasticsearch.host:localhost}") private String host; @Value("${elasticsearch.port:9200}") private int port; @Bean public RestHighLevelClient elasticsearchClient() { return new RestHighLevelClient( RestClient.builder(new HttpHost(host, port, "http")) ); } } ```
创建文档索引服务：
``` @Service public class DocumentIndexService { @Autowired private RestHighLevelClient client; public void indexDocument(ArchiveDocument document) throws IOException { IndexRequest request = new IndexRequest("archive_documents") .id(document.getId()) .source( "title", document.getTitle(), "content", document.getContent(), "author", document.getAuthor(), "createdDate", document.getCreatedDate(), "department", document.getDepartment(), "keywords", document.getKeywords() ); client.index(request, RequestOptions.DEFAULT); } } ```四、存储管理实现
4.1 多级存储策略配置
创建存储策略工厂：
``` @Service public class StorageStrategyFactory { @Autowired private S3StorageService s3Storage; @Autowired private LocalStorageService localStorage; public StorageService getStorageStrategy(ArchiveDocument document) { LocalDateTime createdDate = document.getCreatedDate(); LocalDateTime threeYearsAgo = LocalDateTime.now().minusYears(3); LocalDateTime tenYearsAgo = LocalDateTime.now().minusYears(10); if (createdDate.isAfter(threeYearsAgo)) { return localStorage; // 热存储 } else if (createdDate.isAfter(tenYearsAgo)) { return localStorage; // 温存储 } else { return s3Storage; // 冷存储 } } } ```4.2 MinIO对象存储集成
配置MinIO客户端：
``` @Configuration public class MinioConfig { @Value("${minio.endpoint}") private String endpoint; @Value("${minio.access-key}") private String accessKey; @Value("${minio.secret-key}") private String secretKey; @Bean public MinioClient minioClient() { return MinioClient.builder() .endpoint(endpoint) .credentials(accessKey, secretKey) .build(); } } ```实现S3存储服务：
``` @Service public class S3StorageService implements StorageService { @Autowired private MinioClient minioClient; @Override public String storeFile(String bucketName, String objectName, InputStream inputStream, long objectSize, String contentType) throws Exception { // 确保桶存在 boolean found = minioClient.bucketExists(BucketExistsArgs.builder() .bucket(bucketName).build()); if (!found) { minioClient.makeBucket(MakeBucketArgs.builder() .bucket(bucketName).build()); } // 上传文件 minioClient.putObject( PutObjectArgs.builder() .bucket(bucketName) .object(objectName) .stream(inputStream, objectSize, -1) .contentType(contentType) .build() ); return String.format("%s/%s/%s", endpoint, bucketName, objectName); } } ```五、权限控制系统实现
5.1 RBAC权限模型设计
创建权限相关实体：
``` @Entity @Table(name = "archive_roles") public class ArchiveRole { @Id @GeneratedValue(strategy = GenerationType.UUID) private String id; @Column(nullable = false, unique = true) private String roleCode; // ADMIN, DEPARTMENT_ADMIN, USER, GUEST @Column(nullable = false) private String roleName; @ElementCollection @CollectionTable(name = "role_permissions", joinColumns = @JoinColumn(name = "role_id")) @Column(name = "permission") private Set5.2 Spring Security配置
安全配置类：
``` @Configuration @EnableWebSecurity @EnableGlobalMethodSecurity(prePostEnabled = true) public class SecurityConfig { @Autowired private UserDetailsService userDetailsService; @Bean public SecurityFilterChain filterChain(HttpSecurity http) throws Exception { http .csrf().disable() .authorizeHttpRequests(authz -> authz .requestMatchers("/api/auth/").permitAll() .requestMatchers("/api/documents/upload").hasRole("USER") .requestMatchers("/api/documents/delete").hasRole("ADMIN") .requestMatchers("/api/admin/").hasRole("ADMIN") .anyRequest().authenticated() ) .sessionManagement(session -> session .sessionCreationPolicy(SessionCreationPolicy.STATELESS) ) .addFilterBefore(jwtAuthenticationFilter(), UsernamePasswordAuthenticationFilter.class); return http.build(); } @Bean public JwtAuthenticationFilter jwtAuthenticationFilter() { return new JwtAuthenticationFilter(); } } ```六、系统部署与监控
6.1 Docker容器化部署
创建应用Dockerfile：
``` FROM openjdk:17-jdk-slim WORKDIR /app COPY target/.jar app.jar EXPOSE 8080 ENTRYPOINT ["java", "-jar", "app.jar", \ "--spring.profiles.active=prod", \ "--server.port=8080"] ```6.2 应用配置
application-prod.yml配置文件：
``` spring: datasource: url: jdbc:postgresql://archive-postgres:5432/digital_archive username: archive_admin password: YourStrongPassword123! driver-class-name: org.postgresql.Driver data: mongodb: uri: mongodb://root:YourMongoPassword123!@archive-mongodb:27017/digital_archive?authSource=admin elasticsearch: uris: http://archive-es:9200 minio: endpoint: http://archive-minio:9000 access-key: minioadmin secret-key: minioadmin123 archive: storage: hot-path: /data/hot-storage warm-path: /data/warm-storage cold-bucket: archive-cold-storage ```6.3 健康检查与监控
添加Actuator依赖和配置：
```配置Prometheus监控：
``` management: endpoints: web: exposure: include: health,info,metrics,prometheus metrics: export: prometheus: enabled: true endpoint: health: show-details: always ```七、数据迁移与备份策略
7.1 数据库备份脚本
创建PostgreSQL备份脚本backup.sh：
``` !/bin/bash BACKUP_DIR="/backup/postgres" DATE=$(date +%Y%m%d_%H%M%S) DB_NAME="digital_archive" 备份数据库 docker exec archive-postgres pg_dump -U archive_admin $DB_NAME > \ $BACKUP_DIR/backup_$DATE.sql 压缩备份文件 gzip $BACKUP_DIR/backup_$DATE.sql 保留最近30天备份 find $BACKUP_DIR -name ".gz" -mtime +30 -delete ```7.2 档案数据迁移工具
创建数据迁移服务：
``` @Service public class DataMigrationService { @Autowired private JdbcTemplate jdbcTemplate; @Autowired private MongoTemplate mongoTemplate; @Transactional public void migrateLegacyData(String sourceTable) { // 从旧系统查询数据 List按照以上步骤完成后，系统即可投入生产使用。所有服务通过Docker Compose统一管理，前端可通过Nginx反向代理访问各个微服务接口。
相关文章
