Forum: Compiler Issue: QUOTE-SEMANTICS Subsumes: Issue QUOTE-MAY-COPY References: CLtL p. 55, 78, 86, 143 Issue CONSTANT-COLLAPSING Issue CONSTANT-COMPILABLE-TYPES Issue CONSTANT-CIRCULAR-COMPILATION Category: CLARIFICATION Edit History: V1, 22 Jan 1989, Sandra Loosemore V2, 13 Mar 1989, Sandra Loosemore (discussion) V3, 22 Mar 1989, Sandra Loosemore (suggestions from Moon) Status: Proposal NO-COPYING passed at March 89 meeting. Problem Description: Is it permissible for COMPILE and EVAL to coalesce or copy constants? Are there constraints upon what kinds of objects may appear as constants in code processed by COMPILE or EVAL, similar to those for COMPILE-FILE? CLtL p86 states that (QUOTE ) simply returns . On p55 it is mentioned that the only self-evaluating forms that may be copied are numbers or characters. It is also stated that an implementation is permitted to collapse (or coalesce) EQUAL constants "appearing in code to be compiled" (p78), which is defined to mean self-evaluating forms or objects contained in a QUOTE form (without reference to whether the form is processed by EVAL, COMPILE, or COMPILE-FILE). Because of its nature as a file processor, COMPILE-FILE generally must cause copies of constants to be constructed when the compiled code is loaded. In a number of existing Lisp implementations, COMPILE also causes constant objects to be copied and/or coalesced. There is also at least one implementation where constants are copied by EVAL in some circumstances. In the proposals that follow, "copying" is used to mean the process of constructing an object that is "similar as a constant" (as defined in proposal CONSTANT-COMPILABLE-TYPES:SPECIFY), but not necessarily EQL, to the original. The term "coalescing" is defined in the writeup for issue CONSTANT-COLLAPSING. Proposal QUOTE-SEMANTICS:NO-COPYING: State that copying or coalescing of constants appearing in code processed by EVAL and COMPILE is not permitted; the resulting program must reference objects that are EQL to the corresponding objects in the source code. The constraints on what kinds of objects may appear as constants (described in issues CONSTANT-COMPILABLE-TYPES and CONSTANT-CIRCULAR-COMPILATION) apply only to COMPILE-FILE. Rationale: This proposal is consistent with what many people think of as the "traditional" semantics for QUOTE. It gives users maximum flexibility about what kinds of objects may appear as constants. Proposal QUOTE-SEMANTICS:COPYING-ALLOWED-BUT-NO-CONSTRAINTS: State that copying or coalescing of constants appearing in code processed by EVAL and COMPILE is permitted. Copying or coalescing may only take place when the source code is "promoted" to being a program by EVAL or COMPILE, not at runtime. Function definitions are promoted to being a program when the form enclosing the definition (e.g., a FUNCTION or DEFUN form) is promoted. Any object may validly appear as a constant in code processed by EVAL or COMPILE. The constraints on what kinds of objects may appear as constants (described in issues CONSTANT-COMPILABLE-TYPES and CONSTANT-CIRCULAR-COMPILATION) apply only to COMPILE-FILE. For data types where proposal CONSTANT-COMPILABLE-TYPES:SPECIFY does not define the notion of "similar as a constant", an implementation is permitted to copy objects of that type only if it has extended "similar as a constant" to include that type. Rationale: This proposal is the most consistent with the semantics stated in CLtL. It gives users maximum flexibility about what kinds of objects may appear as constants. Allowing constants to be coalesced or copied has advantages for memory management; for example, constants can be copied to read-only memory that does not need to be garbage-collected. Proposal QUOTE-SEMANTICS:SAME-AS-COMPILE-FILE: State that copying or coalescing of constants appearing in code processed by EVAL and COMPILE is permitted. Copying or coalescing may only take place when the source code is "promoted" to being a program by EVAL or COMPILE, not at runtime. Function definitions are promoted to being a program when the form enclosing the definition (e.g., a FUNCTION or DEFUN form) is promoted. The constraints on what kinds of objects may appear as constants (described in issues CONSTANT-COMPILABLE-TYPES and CONSTANT-CIRCULAR-COMPILATION) apply to EVAL and COMPILE as well as to COMPILE-FILE. Rationale: This makes the rules for handling of constants consistent between EVAL, COMPILE, and COMPILE-FILE. It gives implementors maximum flexibility in handling constants in EVAL and COMPILE. Allowing constants to be coalesced or copied has advantages for memory management; for example, constants can be copied to read-only memory that does not need to be garbage-collected. Current Practice: Implementations in which COMPILE attempts to copy all constants include PSL/PCLS, Utah Common Lisp, and Kyoto Common Lisp. UCL and KCL both implement COMPILE effectively as a combination of COMPILE-FILE and LOAD. In Lucid Common Lisp, constants are not normally copied by COMPILE, but since COMPILE does coalesce constants, it may cause QUOTE to return an object which is not EQL to the object which appeared in the source code. Symbolics Genera has COMPILE copy list, string, non-displaced array, and (I-Machine only) closure constants, but Moon says he thinks this is wrong. There is known to be at least one implementation where expanding the DEFUN macro causes all constants in the body of the function to be copied. Cost to implementors: Proposal NO-COPYING would involve a significant cost in those implementations where constants are now copied or coalesced by EVAL and COMPILE. The aspect that is likely to cause the most problems is that, in some implementations, the garbage collector assumes that constants referenced in compiled code have been copied to read-only storage and do not need to be scanned or relocated. Changing this would require major changes to the internal representation of functions, memory management strategy, and/or the implementation of COMPILE. Some implementations would also require substantial changes to support proposal COPYING-ALLOWED-BUT-NO-CONSTRAINTS. Implementations that would have garbage collection problems under proposal NO-COPYING would have the same problems under COPYING-ALLOWED-BUT-NO-CONSTRAINTS, unless they can define a copying behavior that will correctly handle objects of all possible datatypes. Proposal SAME-AS-COMPILE-FILE has no adoption cost above what is required to support issues CONSTANT-COMPILABLE-TYPES and CONSTANT-CIRCULAR-COMPILATION. Cost to users: Proposals COPYING-ALLOWED-BUT-NO-CONSTRAINTS and SAME-AS-COMPILE-FILE may break some existing programs that assume constants in code processed by EVAL or COMPILE are always EQL to the corresponding objects in the source code. Proposal SAME-AS-COMPILE-FILE may also break existing programs that depend on referencing "undumpable" constants in code processed by EVAL or COMPILE. In both cases, however, the behavior is already nonportable. Both proposals would permit implementations in which these programs now work to continue to provide their existing behavior. Benefits: The semantics of QUOTE are clarified. Discussion: This issue subsumes issue QUOTE-MAY-COPY, which caused a very lengthy debate on the cl-compiler mailing list. This issue relates to conformance requirements. Accepting either of proposals NO-COPYING or COPYING-ALLOWED-BUT-NO-CONSTRAINTS would mean that not all conforming programs could be compiled with COMPILE-FILE. Some people may find this disturbing, particularly since one of the goals of Common Lisp has been to try to eliminate differences in semantics between compiled and interpreted code. Loosemore supports proposal QUOTE-SEMANTICS:SAME-AS-COMPILE-FILE, since it requires essentially no conversion cost for implementors and does not break any user programs that are not already nonportable. JonL White says: Since we have already passed the proposal that permits constants to be "read-only" -- it is an error to modify them -- and have already passed the proposal that allows access to updateable structures -- LOAD-TIME-EVAL -- then there is no excuse for being overly concerned with the storage address of quoted data. People who have mistakenly used structured constants as updatable data should convert over to either LOAD-TIME-EVAL or DEFPARAMETER. Kent Pitman says: The problem is that a lot of copying advocates have been going around trying to use "the need for copying" as leverage for restricting the set of things which I may quote. My view is that it is my write [sic] to quote whatever I want, and it's up to the person who thinks they can do something fun with copying to not get themselves in deeper than they can handle. Jeff Dalton says: I would agree [with Pitman's remarks] too. My only quibble is that it's not just "the need for copying" that's used a lever. "Consistency with file compilation" is also being used as a lever. UCL implements COMPILE by dumping and loading a temporary file using the same mechanisms as COMPILE-FILE and LOAD. Leigh Stoller (one of the UCL compiler implementors) says that, even if this implementation technique is disallowed by the outcome of this issue, they would rather be nonconforming than change the implementation of COMPILE. In addition to the change being a lot of work, he says he thinks that making COMPILE-FILE and COMPILE different would be "really dumb", and that having different conformance requirements for compiled and interpreted code would just encourage people to write programs that can't be compiled correctly.