Introduction to Inferencing

(Also known as a Reading Guide on Knowledge-based Legal Applications)

Last updated 3 March 2001.

= Required reading. All other links indicate optional reading.

The aim of this document is to help you obtain an understanding of the possibilities and limitations of knowledge-based technologies when applied to law.

Particular areas of coverage include:

• The history of Artificial Intelligence and 'expert systems', and the main models of computerised inferencing (drawing inferences) that have emerged.
• The relationship between legal theory (jurisprudence) and computerisation of legal tasks.
• Rule-based reasoning and its applications to statute-based law.
• Case-based reasoning and the problems of applying it (or neural networks or anything else) to case law.
• The relationships between inferencing and the other technologies of hypertext and text retrieval.
• The special issues in web-based legal inferencing, and the opportunities for greater integration with hypertext and text retrieval.
• Automated legal document generation systems - why so little progress?

The most substantial collections of research papers available for free access can be found at:

Links to other research papers, home pages of researchers etc can be found at:

Here are the main starting points:

I have listed below some of Moles' principal arguments (but without doing them justice in a brief description), followed by a brief comment on how such an argument could be answered.

• Isomorphism - Moles gives a damning paraphrase of Bench-Capon's description of how isomorphism is maintained in the Alvey Project, and rightly concludes that it shows considerable insensitivity to the nature of legal interpretation. Answer?: Even if Bench-Capon's is a poor example of isomorphism, how is this a criticism of isomorphism as an ideal? Can it be done better? (see the next section of this guide for more extensive discussion)
• Statutes are interdependent, so it is useless to represent in isolation; if we represented all possibly relevant rules, inferencing engines couldn't cope. Answer?: A good point, there is a practical limitation to the comprehensiveness of any expert system. But the same criticism applies to a textbook, but doesn't stop them being written. Is this a double standard? Isn't it a question of expert judgment as to what other rules must be added;? Also, doesn't the user still need to use discretion or an over-ride capacity ? (This is the `its only a tool' argument.)
• Statutes and the common law are interdependent , so its no use just representing statutes. Moles gives superb example concerning the British National Act (the subject of a famous early legal expert system by Sergot et al) - to understand the Act it is necessary to understand a statue 250 years old and the case law on it at the time. Answer?: There is no way around this but to rely on expert judgement to add some rules (or opportunities for users to make decisions?) reflecting these interpretative issues, as there is no possibility of comprehensiveness!
• Consequentialism - Courts [always?] apply consequentialist reasoning (ie consider the consequences of reasoning in deciding whether the reasoning is adequate), so it is impossible to separate the knowledge representation from the inferencing engine. He give an excellent example of the DIY shop case. Answer?: Perhaps there isn't any, unless you hypothesise meta-rules like the one he quotes; also, although he (via MacCormick) claims that such reasoning is encountered `at every turn', it must be something of an empirical question as to how much this threatens the viability of legal expert systems.
• Words are ambiguous - He correctly chides the logic programmers for silly statements about choosing unambiguous interpretations of legal sources; and for conducting surveys to determine meanings of words. Answers?: He wrongly conflates Susskind's misuse of the `survey' approach with what the logic programmers do, and doesn't seem to appreciate Susskind's `stop here' approach to ambiguity. The answer to this can only be that the ambiguity of language can only be dealt with by leaving some interpretative decisions in the operation of an expert system to the user (while attempting to assist the user to find the correct interpretation), and by attempting to make it explicit where interpretations have been built in to the knowledge-base (part of the idea of isomorphic representation).
• Rules [or groups of them] are not atomistic entities, so isomorphism's claimed advantages in terms of maintainability are fictitious. Answer?: There is a major error of understanding in Moles' thinking that to say that to change one rule is to change the meaning of other rules is somehow a criticism of isomorphism - this is precisely what isomorphism aims to achieve!
• The view that there is no need for legal expertise is obvious rubbish - Moles suggest that if the logic programmers (in the British Nationality Act example) had obtained expert advice it would have destroyed their project, and makes the acerbic comment that `They have used logic programming to model the logical consequences of their own untutored assumptions as to how a statute, dealing with an area about which they have no experience, would be read and used by someone about whom they have no understanding'. No answer needed, Moles is correct that legal expertise is needed.

Moles concludes that there has been no progress by anyone in the previous 20 years! He sees the need to approach with an `open mind, the basic question of the suitability of the legal domain for he development of expert systems'

 It is worth asking 'what is Moles really criticising?' - Is his model of 'logic programming' a straw man? If he is criticising the quest for artificial intelligence in the strict sense, then his pessimism is justified, but not if he is purporting to generalise this into a more overall critique of inferencing systems as useful tools for lawyers. If he is, then the relevant comparison may be a textbook, not a human mind. Perhaps this is a mole-hill not a mountain.

'Isomorphism' in legal knowledge representation, used in the sense of 'creating a well defined correspondence between source documents and the representation of the information they contain [that is] used in the system' [Bench-Capon and Coenen, 1992], has been shown to provide many advantages for the operation of legal inferencing systems (despite dissents such as that of Moles).

 The approach taken in this paper is very similar to that taken by SoftLaw (see following section), who have applied an isomorphic and English-like representation to their development of large commercial inferencing systems.

 Browse the SoftLaw pages, and read the following to obtain an introductory idea of SoftLaw's approach:

 The following older papers concerning SoftLaw are in the printed volume of Developing Computer Applications to Law (B) subject materials Vol # 5 - Case Study: SoftLaw available at the UNSW Law Library Reserve Desk.

• Peter Johnson and David Mead `Legislative knowledge base systems for public administration - Some practical issues' Proc. 3rd International Conference on Artificial Intelligence and Law, ACM Press, 1991
• Surendra Dayal, Peter Johnson & David Mead `Natural language - An appropriate knowledge representation scheme for the administrative domain' SoftLaw Corporation, 1993
• Surendra Dayal et al `Beyond knowledge representation - Commercial uses for legal knowledge bases' Proceedings 4th International Conference on Artificial Intelligence & Law, Amsterdam, 1993, ACM Press
• Softlaw Corporation - `STATUTE Corporate - Component Technologies Guide' (Chapter on STATUTE Expert) Softlaw Corporation, Canberra, 1991

A principal aim of the DataLex Project (1985-1994) was to explore the integration of legal inferencing technologies with hypertext and text retrieval.

AustLII is a proponent of legal inferencing systems via the internet, partly because of the potential it creates for legal inferencing systems to be integrated with large online collections of primary legal materials (like AustLII) which are being updated constantly.

There are many attempts to develop case-based reasoning systems in law, but little information seems to be available on the web. The following notes only discuss one example of a case-based reasoning system applied to law.

These notes deal with various implementations of FINDER, Alan Tyree's implementation of case-based reasoning using 'precedent analysis by nearest-neighbour decision analysis' (PANNDA). FINDER is actually the particular example application using the 'finder's cases', whereas the general approach is more properly called PANNDA. However, the general approach is also referred to below as 'FINDER'.

These notes are derived from Alan Tyree Expert Systems in Law Prentice Hall 1989 Ch 7, pgs 137-44, and from discussions with Alan Tyree. They are, however, my interpretations (GG).

 `Similarity' is the basis of FINDER - `The concept [of similarity] is at the heart of any well-defined notion of precedent.' (eg the maxim `like cases should be decided alike'); PANNDA attempts to model similarity in a `direct' fashion, by a mathematical measure of the presence/absence of key attributes of each case.

 The key assumptions in FINDER are:

• that a group of cases can be identified as relevant to deciding a legal issue;
• that it is impossible to deduce any rule(s) of general application from the cases;
• that it is, however, possible to identify attributes of those cases which are relevant to the Courts' decisions [although exactly how they are relevant can't be reduced to a rule]; for PANNDA, these attributes must also be reduced to Yes / NO form]
• `that we have exercised care and skill in representing the attributes and values of each of the cases at an appropriate level of generality' (Tyree p138) -- ie the application of expertise.

This approach could be relevant where any statute or case simply lists factors which are to be taken into account or `weighed' in reaching a conclusion.

 A summary of the method used in FINDER (in the 'finder's cases' example):

• each of 10 factors identified by expert given a 1/0 ie present/absent in case;
• low variance factors = where there are a preponderance of 1s or 0s in the 8 cases; high variance = where a 4/4 or 3/5 split; they ranged from 7/1 (low) for A4 and A9 to 4/4 (high) for A2; the low variance factors (7/1 etc) are considered to be more significant than the high variance ones ; therefore they are given a lower weighting (VAR) (eg .109), whereas high variance ones have higher weighting (eg .250);
• for each Q, each factor in the test case is compared with the case; if the factors are the same, the score is 0; if different, the score is 1/VAR; the total distance between each case and the test case is the sum of these 10 scores ; so, the more zeros, the lower the scores; and the more difference on low variant factors, the higher the scores.
• the case with the lowest score is regarded as the closest to the test case; those with higher scores progressively further away. ie not simply a matter of which one has the highest no of 0s and 1s the same as the test case.
• The assumption behind the measure of variance (which determines the weight to be given to attributes) is that factors which almost all cases share, but 1 or 2 don't (ie 7/1/ or 6/2) are likely to be the most significant factors determining the outcome of a set of cases. Therefore, if a case differs from the test case in respect of one of these, this should mean it is `further away' than if it merely differed on one factor which is spread uniformly b/w all the cases (ie 4/4/ or 3/5).
• The measure of variance is not correlated with the outcomes of the cases; ie if only two cases in a set of 10 have a Y value for variable A, A is regarded as more important than B which has a 50/50 Y/N split -- but the system doesn't know if there is a significant correlation b/w a Y answer for variable A and a particular outcome for the precedent. Tyree explains (verbally) that there are usually too few cases to make such a correlation in a way that will be statistically significant [otherwise, wouldn't we just represent such a correlation as a rule??]. The justification for his approach is that the cases, and the factors described within them, are not drawn at random, but are selected by the expert because of their significance. Therefore, variables with 4/4 or 5/3 splits actually separate the cases into groups more effectively, so `why would an expert include a 7/1 variable unless it is particularly significant'. He agrees this is contentious.
• The outcome of a FINDER consultation is that a report is generated with identifies the 'nearest' case (to be followed) and the nearest case with the opposite outcome (to be distinguished), and the similar and distinguishing facts for both nearest case and nearest opposite case. It is a simulation of a legal argument from a statistical process. There is an attempt at recognition of hard cases, when it identifies that there are two 'nearest' cases (ie the same distance away from the test case) with opposite outcomes.

Some imitations of FINDER:

• It is limited to true / false values (the wysh version also includes 'uncertain')
• It makes no distinction between the hierarchical precedential value of cases. It is questionable whether this is a criticism -- perhaps it should only be used where there are a `flat' set of cases, where no particular case is regarded as more authoritative than others.
• It does not emulate human legal reasoning in reaching its conclusions [as far as we know]. Tyree attempts to show that it does produce a similar result, however.

One of the best-known current Australian research projects in this area is in the domain of property settlements in family law, legal decision support system built by the Database Research Laboratory at Melbourne's LaTrobe University Computing Institute.

For papers generally concerning case-based reasoning in law see:

There is relatively little available on automated document generation in law, despite it seeming to be one of the most obvious commercial implementations of inferencing technologies to law.

One of the most active fields of AI research in law is to adapt AI techniques to improve legal text retrieval.