º¬Ðß²ÝÉçÇø

AI is rapidly permeating the judicial sector, with applications ranging from intelligent document generation to case element analysis. AI is now integrated across the entire litigation process. The global legal technology market has surpassed USD10 billion, and investment in the digitalisation of China¡¯s judicial system is accelerating each year. However, while this transformation brings about an efficiency revolution, it also introduces systemic risks.

Data security, privacy risks

The application of AI in litigation services relies on models trained with vast amounts of legal data, including judicial decisions, case files, contract terms and client privacy information. Many legal sector-specific large models require access to enterprise intranets or law firm databases for local deployment.

Inadequate security measures may result in data leaks or malicious attacks. Furthermore, if AI-generated legal documents are not properly anonymised, they may expose clients¡¯ commercial information or secrets, leading to legal disputes. Based on practical research, the following strategies are recommended.

Design multi-layered encryption and access control mechanisms. When legal sector-specific large models access intranets or local databases, end-to-end encryption should be adopted alongside a strict access control system. For example, blockchain technology can be used to add an ¡°anti-counterfeiting chain¡± to electronic case files, ensuring immutability of data flows. From a cost perspective, a hybrid single-archive approach may be considered, retaining only key paper originals, with other documents generated and circulated through encrypted electronic case file systems, reducing the risk of data leakage.

Anonymise and desensitise data. During the privacy data governance stage of model training, a graded and classified data security management mechanism should be implemented. For data containing personal privacy or commercial secrets, a standardised desensitisation process covering the entire data lifecycle should be established.

Specifically, standardised case handling procedures should be constructed with modular management of information collection, cleansing and labelling. Intelligent data collection systems should be deployed using pre-set rule engines to automatically identify and filter sensitive fields (including but not limited to ID numbers and financial account information), and differential privacy techniques should be used to anonymise structured data, resulting in compliant datasets that retain only core case elements.

It is noteworthy that some courts have begun piloting ¡°element-based intelligent adjudication systems¡± to enable intelligent extraction of case elements and automatic aggregation of legal facts. Their experience in data element extraction and structuring offers valuable reference.

Reliability and verification

From a legal technology risk management perspective, the main risks of current AI judicial assistance tools centre on technical reliability and data authenticity. According to research, the risk exposure of AI in litigation practice is mainly reflected in the following aspects.

Inherent flaws in generative algorithms lead to inaccurate legal outputs. Both leading international models such as ChatGPT and mainstream domestic models such as DeepSeek exhibit varying degrees of algorithmic factual distortion, such as fabricating non-existent judicial interpretations or citing incorrect case numbers. In addition, the lag in updating training data further amplifies these risks. Mainstream legal databases typically update case law with a delay of three months or more, making it difficult for AI to adapt to the latest judicial developments.

AI has structural limitations in reasoning about complex legal scenarios. When handling complex transaction structures such as multi-layered SPVs, convertible earn-outs, multi-level cross-border guarantees or technical unfair competition disputes, AI¡¯s accuracy in interpreting non-standard contract terms, analysing key case points and identifying true legal relationships drops sharply, making it difficult to effectively identify special rights and obligations.

This technical bottleneck is particularly evident in dispute resolution involving complex commercial co-operation, financial derivatives, cross-border M&A, and emerging technology development.

Judicial fairness and tech bias

The deep application of AI in judicial proceedings also brings risks of algorithmic power distortion and fairness. A ¡°computing power gap¡± is emerging in the legal services market, with leading law firms leveraging capital, human resources and information advantages to build algorithmic and data barriers, creating localised, closed algorithmic ecosystems.

Small and medium-sized firms are unable to match this level of investment. In the absence of robust regulation, such barriers will become entrenched, resulting in de facto technological monopolies.

Of particular concern is the coupling risk of the black box algorithm and implicit bias in historical data, which may trigger a ¡°bias iteration loop¡± and systemic risk. Institutions with technological monopolies may develop algorithmic models that absorb non-standardised factors from historical judgments, embedding structural bias into automated decision systems.

These products, endowed with academic authority, are perceived as ¡°technically neutral¡± and gain information dissemination advantages through algorithmic recommendations on online platforms. The process of digital reconstruction of legal facts becomes dominated by algorithmic logic, and the analytical models of leading institutions may distort the standards for identifying case elements and the mechanisms for assigning their relative weight, resulting in systemic shifts in the judicial cognitive framework.

This expansion of technological power produces two negative effects: first, the ¡°authoritative conclusions¡± output by algorithms become de facto standards through public feedback, gradually eroding judicial discretion; second, the technology gap exacerbates the imbalance of capabilities among litigants, making it difficult for parties to effectively challenge algorithmic decisions due to a lack of interpretability, ultimately undermining the procedural justice foundation of the adversarial litigation system.

In the author¡¯s view, only when technology is harnessed in service of justice¡¯s core tenets can AI become a genuine digital ally, rather than a Trojan horse undermining the foundations of the rule of law.

Ethan Zhang is a senior partner at Joint-Win Partners.