Index
Bold numbers indicate definitions of terms.
Symbols and Numbers
_ (underscore), in Ruby, 437
; (semicolon), as a separator, 316, 320, 336
: (colon), in Ruby, 78, 439, 443–444
? (question mark
implementing, 258
in lexers, 234
in Recursive Descent Parser, 247
/ (ordered alternative operator), 233
in Recursive Descent Parser, 246
. (period), in method calls, 374
.. (range operator), 232
^ operator, in Regex Table Lexer, 240
~ (up-to operator), in ANTLR, 232
'...' (single quotes), 321
"..." (double quotes)
in lexers, 320–322
in XML, 101
using for Alternative Tokenization, 101
{...} delimiters
in C#, 408
in DSLs, 42
in Javascript, 322
in lexers, 322
{{...}} delimiters (Java), 395
@ (at), in Java, 446
* (asterisk)
in ANTLR, 302
in Recursive Descent Parser, 247
in Ruby, 421
Kleene, 293
&& operator (C#), 461
# (hash mark)
in comments, 63
in template commands, 544
+ (plus sign)
actions for, 259–260
as multiplicity marker, 235
in Parser Combinator, 258
in Recursive Descent Parser, 247
<...> (angle brackets), in XML, 101
| (alternative operator)
actions for, 259
implementing, 257
in C, 257
in lexers, 233–234
in Recursive Descent Parser, 246
A
Access. See Microsoft Access
binding, 529
embedding, 270–271
in Parser Combinator, 259–260, 268
in Tree Construction, 292–297
invocations for, 260
keeping small, 126
Active Record (Ruby), 428, 430, 434
Adaptive Model, 18, 81–82, 115–116, 161, 256, 402, 487–493
embedding imperative code into, 489
Closure, 401
Production Rule System, 492
State Machine, 529
visualizing, 491
Algol, 229
alternative computational model, 18, 113–120, 487–493, 505–530
See also Adaptive Model, Dependency
Network, Production Rule System, Static Machine
Alternative Tokenization, 94, 222–223, 319–326, 340
quotes in, 101
using with:
Foreign Code, 101, 309, 317, 326
Syntax-Directed Translation, 326
implementing, 246
unordered, 233–234
with recursion, 235
Annotation, 32, 84–85, 445–454
custom syntax for, 449
parameters in, 446
processing, 447–451
using with Semantic Model, 447
validation, 448–454
Ant, 36, 108, 110, 119, 157, 508
syntactic analyzer in, 274–276
ANTLR, 92, 97–99, 102, 141, 152, 230, 241, 270–280
AST in, 283
calculator in, 306–307
Dependency Network in, 168–170
grammar files in, 549–552
“Hello World” in, 272–280
lexer in, 222
negation operator in, 326
nested tokens in, 318
Notifications in, 195–198
parsing in, 286–288
recognition exceptions in, 196–197
return types and variables in, 237
rules in, 237
superclasses in, 552
syntactic analyzer in, 274–276
top-level rule in, 330
up-to operator in, 232
using with Foreign Code, 316
Applescript, 42
ASCII, 429
ASP (Active Server Pages), 125, 540
associative arrays. See Literal Map
AST (Abstract Syntax Tree), 226, 281–297
creating in ANTLR, 283
See also syntax tree
attributes, 84
autocompletion, 22, 107, 358, 363, 382–384, 467, 469, 521
Automake, 157
automata theory, 221
Awk, 28
B
backward chaining, 515
Behavior Verification, 149
Bell, 29
blinkered abstraction problem, 39
blocks. See Closure
BNF (Backus-Naur Form), 79, 90–92, 220, 229–238
code actions in, 283
using with:
Embedded Translation, 301
Parser Generator, 270
vs. Parser Combinator, 261
browsers, 448
Buck, Jamis, 434
build process, 574
C
function pointers in, 82
generating, 46
macros in, 184–186
pointer arithmetic in, 558
standard library of, 565
C preprocessor
macros in, 53
template commands in, 544
C# language
annotations in, 446
anonymous delegates in, 399–400
autocompletion in, 521
conditions in, 457–466
configuration files in, 12
Decision Table in, 497–503
explicit class methods in, 351
extension methods in, 481
hashmaps in, 167–168
implicit conversion operator in, 376
in Dependency Network, 508–512
lambdas in, 81–82, 456, 460, 519, 521
Literal Extension in, 483–484
Nested Closure in, 408–409
Notifications in, 194–195
object initializers in, 365
parse tree in, 456
Parse Tree Manipulation in, 83, 191, 456
periods at the start of the line in, 374
progressive interfaces in, 382–384
property syntax in, 381–382
recurring events in, 366
SQL queries in, 83
statically typed symbols in, 128
string arguments in, 521
using for Context Variables, 176–178
validations in, 517–521
varargs function calls in, 77
C++ language
parsing, 226–227
templates in, 53
calendar, 345–347
multiple builders for, 348–350
callouts, 539–546
CASE (Computer-Aided Software Engineering) tools, 141–142
CFG (Context-Free Grammar), 233
chaining
methods. See Method Chaining
rules, 117
Chomsky hierarchy, 96–97
Chomsky, Noam, 96
Class Symbol Table, 88, 107, 168, 173, 467–475, 511
in Java, 469
statically typed, 469–475
using with:
Dependency Network, 509
Expression Builder, 467–468
Semantic Model, 468
classes
collaborating, 573
generating from a data schema, 573–577
initializing fields of, 471–472
partial (C#), 573
private, 445
public, 445
serializing, 445
using fields as a symbol table, 475
Closure, 80–82, 150, 188, 359, 397–402, 489
in C#, 81–82, 456, 460, 519, 521
multiple, 401
nested. See Nested Closure
using with:
Adaptive Model, 401
Foreign Code, 491
code actions. See actions
code coloring. See syntax highlighting
code generation, 19–21, 46–47, 92, 121–128
and handwritten code:
intermixing with, 448
separating from, 126–127, 535, 571–577
and Semantic Model, 21, 128, 130, 162, 533–538
by Embedment Helper, 548, 552–554
dynamic, 452–454
for static languages, 100, 566
keeping in separate files, 573
preparsing, 128
readability of, 127–128, 545, 563, 566
reasons for, 121–122
styles of, 21
target audience of, 558
target environments for, 121
Collecting Parameter, 194
collections, 540
combinatorial explosion, 235
combinators, 256
functional style of, 260 See also Parser Combinator
Command, 489
command objects, 82
command-query APIs, 16, 29, 69, 343–350
bad, 38
documenting classes in, 29
naming methods in, 70
vs. DSLs, 37
command-query separation, 70, 374
comments
generating, 538
in C, 234
in Parser Generator, 274
compilation, 19
type checking during, 469
compiled languages, 448
composed regex, 209
computational models
alternative. See alternative computational model
conditionals, 113, 405, 513–526
overlapping, 529
configuration files, 30–31
Construction Builder, 51, 176, 179–181, 348, 353, 378
adding lifecycle controls to, 180
full delegation in, 381
grouping, 180
uses of, 180
using with Semantic Model, 353–354
context, 540
Context Variable, 49, 52, 73–74, 76, 175–178, 237, 271, 317, 351–353, 361, 369, 371, 376–379, 381, 433
and debugging, 176
and parsing, 300
using with:
Function Sequence, 351–355, 358
Nested Closure, 404–405
Nested Function, 359
Parse Tree Manipulation, 466
control flow operators, 245
CSS (Cascading Style Sheets), 150–151
specifying colors in, 53, 184–185
using macros with, 53, 184–185
CSV (comma-separated values), 496, 574
Custom Assertion, 56
D
DAG (directed acyclic graph), 505
Data Mapper, 160
data schemas, 573–577
databases, 115
accessing, 573
desktop tools for, 136
mapping, 448
Davidson, James Duncan, 110
and code readability, 566
and line numbers, 375
and mistaken expansions, 186
diagnostics for, 563
of Production Rule System, 517
trace statements for, 126
using comments for, 274
with Adaptive Model, 492
with Context Variables, 176
with pointer arithmetic, 558
Decision Table, 114–118, 156, 495–503
in C#, 497–503
using with:
Delimiter-Directed Translation, 502
Semantic Model, 496–502
vs. Production Rule System, 497
deferred evaluation, 80–81, 189
delimiter characters, 90, 201, 321
Delimiter-Directed Translation, 91, 97, 107, 177, 201–218, 222, 565
syntactic indentation in, 339
using with:
complex languages, 204
Decision Table, 502
Newline Separators, 333
regular expressions, 90, 202, 209, 214
vs. Embedded Interpretation, 202
vs. Embedded Translation, 202
vs. Syntax-Directed Translation, 92, 106, 204–205, 227
vs. Tree Construction, 202, 286
Dependency Network, 18, 36, 108–109, 119, 156–158, 505–512
in C#, 508–512
in external DSLs, 168–170
invoking vs. executing tasks in, 507
missed prerequisites in, 507
parsing, 511–512
product-oriented vs. task-oriented, 506–508
targets in, 506
unnecessary builds in, 507
using with:
Class Symbol Table, 509
Semantic Model, 509–511
Design by Contract, 135
diagrams
and State Machine, 136
describing in a textual format, 154
in projectional editors, 136, 141
dictionaries. See Literal Map
do...end delimiters (Ruby), 401, 406, 408
doesNotUnderstand method (Smalltalk), 428
DOM (Document Object Model), 48, 93, 101–102
domain experts, 34–35, 108, 116, 479, 496–497
domain languages, 32
vs. Semantic Model, 44, 134, 160
Domain-Driven Design principle, 5
DSL processor, 121
DSL scripts
executing, 472
DSLs (Domain-Specific Languages)
advantages of, 18
building cost of, 106–107
changeability of, 21
clarity of, 33
common conventions in, 42
composing, 111
compositional, 161
computational, 161
custom syntax of, 13, 15–16, 28
defining, 27–32
embedded. See internal DSLs
embedding languages in. See Foreign Code
extending, 100
external. See external DSLs
fragmentary, 32, 84, 387, 392, 415
IDE for. See language workbenches
internal. See internal DSLs
layering on top of another, 111, 151
learning cost of, 37
limitations of, 28, 151, 309, 339
limited expressiveness of, 13, 27–30, 33, 110, 121
maintenance cost of, 37–38
migrating, 64–66
problems with, 36
readability of, 310–311, 360, 363, 469, 471
semantic model of, 29
semantic predicates for, 227
syntactic structure of, 269–280
uses of, 33–36
vs. general-purpose languages, 28, 30, 42
vs. property lists, 101
wrapping third-party libraries with, 33, 38
DTD (Document Type Definition), 102
DTO, 574
Dynamic Reception, 86–87, 427–444
limitations of, 429
using with:
Method Chaining, 428
Object Scoping, 428
Textual Polishing, 428
and external DSLs, 110
evaluate text at runtime in, 477
Foreign Code in, 100
Literal Maps in, 419
overriding unknown methods in, 428
runtime errors in, 427
E
EBNF (extended BNF), 229–238
intermediate rules in, 235
Eclipse, 141
Egge, Brian, 39
Emacs, 102
filtering, 514
searching, 457–466
embedded DSLs. See internal DSLs
Embedded Interpretation, 93, 226, 236–237, 305–307, 331
and Parser Generator, 271
using with Nested Operator Expression, 328
vs. Delimiter-Directed Translation, 202
vs. Syntax-Directed Translation, 276
Embedded Translation, 93, 101, 148, 168, 196, 226, 236–237, 261, 284, 299–304, 549
actions in, 259
advantages of, 300
and Parser Generator, 271
disadvantages of, 300
in Recursive Descent Parser, 248
populating Semantic Model with, 284
using with:
BNF, 301
Embedment Helper, 301
vs. Delimiter-Directed Translation, 202
vs. Syntax-Directed Translation, 276–278
vs. Tree Construction, 93, 284, 300
Embedment Helper, 126, 148, 169, 195, 238, 277, 316, 547–554
advantages of, 549
generating code with, 548, 552–554
naming for, 551
using with:
Embedded Translation, 301
Foreign Code, 310, 312, 315, 547–549
Templated Generation, 125, 540, 544, 548
encapsulation, 563
end-of-statement rule, 334
enums
for Literal Map keys, 419
for statically typed symbols, 172–174, 469
cost of, 63
semantic, 63–64
special methods for, 428
syntactic, 63
error messages, 62
errors
collecting, 193–198
compilation, 427
reporting in ANTLR, 277–278–280, 286
runtime, 427
escaping mechanism, 321
Evans, Eric, 67
Excel. See Microsoft Excel
exceptions, 194
Execute-Around Method, 188
expectations, 149–150
Expression Builder, 45, 71–73, 81, 107, 149, 171–172, 180, 343–350, 351, 370, 373, 375, 377, 382–383, 386–387, 406, 411, 428, 468, 472, 481–482
composite, 344
multiple, 344, 348–350, 409–411
separating from Semantic Model, 71, 344
testing, 344
using with:
Class Symbol Table, 467–468
Literal Extension, 481–482
Method Chaining, 344
Nested Function, 344
vs. parsers, 71
external DSLs, 15, 28–29, 89–103
boundaries of, 30–31
changing, 64
comments in, 63
composing, 111
Dependency Network in, 168–170
disadvantages of, 22
embedding in a method name, 428
flexibility of, 245
learning cost of, 105–106
limitations of, 109
runtime configuration of, 110
Symbol Table in, 168
Textual Polishing in, 478
vs. internal DSLs, 45–47, 89, 105–112, 478
F
filtering, 397–402
finishing problem, 375–376
finite-state machine, 96
FIT (Framework for Integrated Test), 31, 155–156
Fitnesse, 155
Flex, 322
fluent interfaces, 16, 30, 67–70, 107, 343–350
vs. command-query APIs, 70, 72
Foreign Code, 100–101, 109, 111, 195, 226, 236, 270, 309–318, 320, 340, 540
advantages of, 311
and Semantic Model, 310–315, 318
compiled, 310
interpreted, 310
parser for, 315–318
using with:
Alternative Tokenization, 101, 309, 317, 326
ANTLR, 316
Closure, 491
dynamic languages, 100
Embedded Translation, 299–301, 315
Embedment Helper, 310, 312, 315, 547–549
Parser Generator, 270–271
Repository, 316
Syntax-Directed Translation, 126
forms
embedding Javascript in, 520
submitting, 449
validation, 448–449
forward chaining, 515, 521, 523–524
forward references, 302
full delegation, 381
function pointers, 82
Function Sequence, 49, 68–69, 72–76, 79–80, 150, 170, 351–355, 359, 376, 385–386, 411
using with:
Context Variable, 351–355, 358
Method Chaining, 352
Nested Closure, 352, 394, 404–407, 411–412
Nested Function, 352
Object Scoping, 352–353
Production Rule System, 523, 526
vs. Literal List, 352
functional programming, 163
functions, 72–77
anonymous. See Closure
child expressions of, 403–415
combining, 72
keyword arguments in, 419
varargs parameters in, 77, 358, 361, 417–419, 421
vs. Macros, 185–186
See also methods
G
general-purpose languages, 27
code generation with, 92
learning cost of, 37
using as DSLs. See internal DSLs
generated code. See code generation
Generation Gap, 127, 278, 571–577
using with ANTLR, 278–280
gensyms, 190
ghetto language problem, 38–39
Gimp, 15
global functions. See functions, global
global state, 385
GLR parser, 99
putting code actions into, 547
grammars, 16, 29, 49–50, 79–80, 90, 95–96, 219–220, 227, 229–238
and parse tree, 135
changing, 99
context-free, 97–98
context-sensitive, 97
modular, 339–340
vs. schema definition languages, 135
Grant. See secret panel controller
graphs, 147–148
error messages in, 62
Greenspun form, 422–425
GUI design, 139
H
handwritten code, 21
intermixing, 448
separating from generated code, 126–127, 535, 571–577
hashmaps. See Literal Map
Hibernate. See HQL
hierarchic context, 302
hierarchic structure, 376
HQL (Hibernate Query Language), 28, 151–152, 325
HTML (Hypertext Markup Language), 552
I
IDE (Integrated Development Environment) advantages of, 107
build process in, 574
compile-time type checking in, 469
for DSLs. See language workbenches
refactoring capabilities of, 572
representations in, 138–139
support of, 122, 377, 382, 466, 469
type-aware autocompletion in, 467, 469
illustrative programming, 24, 130, 138–140
IMAP (Internet Message Access Protocol), 457, 466–466
immutable objects, 179
incremental migration, 65
inference engine, 514
informational messages, 194
INI files, 101
using for Context Variables, 176–178
inline nesting, 80
input token buffer, 248, 257–258, 260, 264–265
instance_eval method (Ruby), 386, 392–394, 405, 407, 412–415, 420, 422, 440, 442, 479
instance initializers, 150, 172, 386, 394–395
integration tests, 62
Intentional Software, 136, 140
Intentional Workbench, 136, 140
interfaces
command-query. See command-query APIs
fluent. See fluent interfaces
marker, 447
operational, 161
population, 161
progressive, 149, 377, 382–384, 519
published, 64
internal DSLs, 15, 28–29, 67–88, 344
boundaries of, 29–30
changing, 64
composing, 111
fragmentary, 32
learning cost of, 105–106
Macros in, 184
making little parse trees in, 438
mixing in host languages, 108–109
naming methods in, 30
parsing, 45–47
vs. external DSLs, 45–47, 89, 105–112, 478
interpretation, 19
invalid input tests, 59–61
J
jargon, 31
Java
actions in, 260
annotations in, 84, 445–446, 449
as a DSL, 15
building languages in, 157
built-in date and time classes in, 346
Class Symbol Table in, 469
configuration files in, 12
Dependency Network in, 168–170
generated and handwritten code in, 126
generating:
classes from a data schema, 573–577
hashmaps in, 167–168
“Hello World” in, 272–280
instance initialization in, 150, 172, 386, 394–395
mapping classes to SQL queries in, 151–152
periods at the start of the line in, 374
setter methods in, 374
static import in, 73, 352, 360, 371, 378
statically typed symbols in, 128
varargs function calls in, 77
curly brackets in, 322
embedding in forms, 520
integrating with Java, 312, 314
validation with, 448
JetBrains, 140
Johnson, Ralph, 115
JSON (JavaScript Object Notation), 103
JSP (JavaServer Pages), 125, 540
K
keywords, 75
and method naming, 480
date-oriented vs. string-oriented, 461
Kleene operators, 231–232, 293
L
LALR parser, 99
lambdas. See Closure
language cacophony problem, 37
language workbenches, 22–24, 28–29, 112, 129–143, 469
boundaries of, 31
editing systems for, 136–138
using with Semantic Model, 142
lay programmers, 138
lexers, 148, 204, 221–223, 230, 232–233, 239–244
comments in, 222–223, 241, 335
implementing, 239
missing rules for some tokens in, 274
operators in, 223
preprocessing text for, 339, 478
quoting in, 320–322
regular expressions in, 221, 223
separating from syntactic analysis, 94–95
separators in, 320, 324–325, 333–336
states of, 322–324
vs. parsers, 319
whitespace in, 222–223, 241, 286, 319, 325
See also Regex Table Lexer
lexical state, 322–324
line continuation characters, 202, 208
line ending characters, 90, 202, 231, 286
duality of code and data in, 488
Macros in, 53, 82–83, 184, 188–191, 456
Nested Functions in, 75
Parse Tree Manipulation in, 83, 191, 456
symbols in, 166
tagging function names with data in, 447
list operator
actions for, 259–260
in ANTLR, 302
lists
literal arrays, 417
literal collections, 77–79
Literal Extension, 85, 375, 481–484
in C#, 483–484
using with Expression Builder, 481–482
Literal List, 77, 79–80, 390, 417–418, 420, 422–423
in Lisp, 78
nested, 424
using with:
varargs function calls in, 417, 419
vs. Function Sequence, 352
Literal Map, 75, 77–80, 170, 366, 419–425
arguments of, 359
in Ruby, 419–422
using with:
Nested Function, 358
Symbol Table, 167–168
LL parsers. See parsers, top-down
loader class, 288
loops, 113
LR parsers. See parsers, bottom-up
M
M language, 141
m4 macro processor, 184
machine states, 528
parameters in, 185
syntactic, 53, 184, 186, 188–192
textual, 52, 184–188, 192, 478
vs. functions, 185–186
Make, 36, 108, 119, 156–158, 508
maps
multiple, 167
nested, 123
single, 166
MathCAD, 21
Maven, 157
MetaCase, 140
meta-models, 131
Meta-Programming System Language Workbench, 31
Method Chaining, 48, 68–70, 72–76, 78–80, 85, 149–150, 370, 373–384, 409, 420, 465–466
and hierarchic structure, 376
end methods in, 479
evaluation order of, 357–359
finishing problem in, 375–376, 380
mandatory elements in, 377
readability of, 381
using with:
Dynamic Reception, 428
Expression Builder, 344
Function Sequence, 352
Nested Closure, 404–405, 409–411
Production Rule System, 523, 526
vs. Nested Function, 381
method_missing method (Ruby), 428, 433–434
Method Object, 209
methods, 72
adding to program literals, 481–484
automatic delegation to another object, 428
chaining. See Method Chaining
context validity of, 377
extension (C#), 481
formatting, 374
global, 85
naming, 30, 70, 137, 374, 428–434, 437, 447, 473, 480, 548
overriding missing. See Dynamic Reception
protected, 127
public, 445
unknown, overriding, 427–444
validation, 577
vs. parameters, 370
See also functions
Meyer, Bertrand, 69
Microsoft Excel, 15, 25, 31, 497
Microsoft Office, 140
Model Ignorant Generation, 122–125, 127, 541, 567–570
vs. Model-Aware Generation, 122–124, 556, 568
model-assisted source editing, 137
Model-Aware Generation, 122–124, 142, 535, 555–566
using with:
Semantic Model, 555–566
Transformer Generation, 535–538
vs. Model Ignorant Generation, 122–124, 556, 568
model-based migration, 65
MPS (Meta-Programming System), 140
multiple evaluation, 186, 190, 192
multiple passes, 302
multiplicity symbols, 231–232, 234
N
negation operator (ANTLR), 326
Nested Closure, 80–82, 87, 109, 150, 154, 158, 376–377, 394, 402, 403–415
explicit arguments in, 411–412, 415
in C#, 408–409
language support of, 405
multiple, 405
using with:
Context Variable, 404–405
Function Sequence, 352, 394, 404–407, 411–412
Method Chaining, 404–405, 409–411
Production Rule System, 523
vs. Nested Function, 393, 403–405
Nested Function, 48, 74–77, 79–80, 87, 149, 191, 357, 359–371, 376, 385–386, 420–421
arguments of:
checking, 363–365
labeling, 358
multiple different, 361–363
disadvantages of, 359–360
evaluation order of, 357–359
in Lisp, 75
in Ruby, 403
mandatory clauses in, 377
using with:
Context Variable, 359
Expression Builder, 344
Function Sequence, 352
Literal Map, 358
Object Scoping, 359, 366, 369, 371
vs. Method Chaining, 381
vs. Nested Closure, 393, 403–405
Nested Operator Expression, 99, 249, 306–307, 327–331
using with Embedded Interpretation, 328
Newline Separators, 95, 222, 320, 333–336
using with:
Delimiter-Directed Translation, 333
Syntax-Directed Translation, 333–336
noise. See syntactic noise
Notification, 58–59, 193–198, 515
in ANTLR, 195–198
in C#, 194–195
parsing, 195–198
using with Production Rule System, 518
O
object initializers, 365
Object Scoping, 73–75, 81, 149–150, 171–172, 361–362, 385, 387–395, 409, 415, 420, 440, 470, 479, 519, 522
using with:
Dynamic Reception, 428
Function Sequence, 352–353
Nested Function, 359, 366, 369, 371
Production Rule System, 525
Observation, 516–517
“Old MacDonald Had a Farm” song, 75, 80, 357, 359
OMG, 141–142
OO (object-oriented) languages
command objects in, 82
domain model of, 134
generated and handwritten code in, 126–127
imperative computational model of, 113
naming classes in, 471
train wrecks in, 68
operators in lexers, 223
optional operator
implementing, 258
in Recursive Descent Parser, 247
output production, 226
P
Packrat, 98
PARC (Palo Alto Research Center), 140
Parr, Terence, 339
parse tree, 226
and grammar, 135
immutable, 456
walking, 456
See also syntax tree
Parse Tree Manipulation, 82–83, 190–191, 455–466
in Ruby, 456
using with Context Variable, 466
Parser Combinator, 91–92, 97, 99, 220, 226, 249, 255–268, 270
actions in, 259–260
composite, 258
downsides of, 261
implementing, 261
input of, 256
output of, 256
recognition in, 256–260
vs. BNF, 261
vs. Parser Generator, 261
vs. Recursive Descent Parser, 256, 261
Parser Generator, 46, 63, 91–94, 96–100, 102–103, 105–106, 111, 129, 220, 223, 225–226–227, 230, 232, 234, 236–238, 241, 245–246, 248–249, 255, 261, 269–280, 283–284, 292, 299–300, 302–303, 311, 319–322, 326–328, 330, 334, 340
and Embedded Interpretation, 271
and Embedded Translation, 271
and Tree Construction, 271
code actions in, 236–238
disadvantages of, 272
embedding actions, 270–271
multiplicity symbols in, 232
using with:
BNF, 270
Foreign Code, 270–271
vs. Parser Combinator, 261
parser objects, 255–268
parsers, 47–49, 201–218, 223–226, 239
bottom-up, 98–99, 238, 327–329
error handling in, 63
for Decision Table, 496, 502–503
for Dependency Network, 511–512
for state machines, 215–218
for XML, 102
generating, 275
top-down, 98–99, 235, 245–254–255, 261–268, 306, 329–331
See also Recursive Descent Parser
vs. Expression Builder, 71
vs. lexers, 319
whitespace in, 202
ParseTree (Ruby), 456
and Context Variables, 175, 300
external DSLs, 45–47, 89–92, 110
multiple together, 101
flexibility in, 162
internal DSLs, 45–47
output of, 92–94
textual transformations prior to, 183
PEG (Parsing Expression Grammar), 97, 233–234
performance, 166
PIC, 154
PL/1, 222
pointer arithmetic, 558
polymorphic lines, 203
precedence rules, 328
precision, 369
printf function (C), 540
procedures. See functions
Production Rule System, 18, 117–118, 136, 489, 513–526
debugging, 517
using with:
Adaptive Model, 492
Nested Closure, 523
Notification, 518
Object Scoping, 525
Semantic Model, 519–521
vs. Decision Table, 497
programming languages, 27
domain-specific. See DSLs
dynamic. See dynamically typed languages
encodings of, 429
general-purpose. See general-purpose languages
interpreted, 110
object-oriented. See OO languages
static. See statically typed languages
templating, 540
with keyword arguments, 75
projectional editing, 136–138, 139, 141
property lists, 101
push-down machine, 97
Q
Quadrant, 141
R
R language, 30
Ragel, 97
Rails, 28
See also Ruby on Rails
Rake, 157
range operator, 232
recurring events, 366
recursion, 235
Recursive Descent Parser, 91–92, 97, 99, 220, 226, 245–254, 256, 261, 270, 423
advantages of, 261
shortcomings of, 249
simplicity of, 249
vs. Parser Combinator, 256, 261
refactoring
in IDE, 572
safe, 137
regex API, 243–244
Regex Table Lexer, 94, 223, 239–244, 246, 250, 256
regular expressions, 28, 30, 32
using with:
Delimiter-Directed Translation, 90, 202, 209, 214
Regex Table Lexer, 239–244
Textual Polishing, 477
Relax NG, 102
repetition operator (Recursive Descent Parser), 247
Repository, 316
reset block, 267
reset events, 4, 8, 288, 297, 303, 475, 563
explicit, 8
Roberts, Mike, 29
Row Data Gateway, 573
arrays in, 456
class methods in, 451–452
dynamic code generation in, 452–454
Dynamic Reception in, 86
Greenspun form in, 422–425
instance evaluation in, 386, 392–394, 405, 407, 412–415, 420, 422, 440, 442, 479
keywords in, 480
Literal Extension in, 85
Literal Lists in, 77–78, 417, 420–422
Literal Maps in, 419–422
Macros in, 187
Nested Function in, 403
Parse Tree Manipulation in, 83
parsed method names in, 430–434
ParseTree library in, 456
periods at the end of the line in, 374
populating Semantic Model in, 35
ranges in, 446
Symbol Table in, 170–171
symbols in, 78, 87, 166, 419, 439, 443–444
Textual Polishing in, 478–480
varargs arguments in, 421
DSL expressions in, 85
metadata in, 84–85
rule engine, 514
rules, 513–526
backward chaining, 515
forward chaining, 515, 521, 523–524
sets of, 515
validation, 516–521
See also Production Rule System
S
salience, 514
SASS, 151
SAX (Simple API for XML), 48, 93, 101
scanners. See lexers scheduler, 514
schema definition languages, 131, 135
schemas, 102
screen layouts, 154
scripts
testing, 61–62
visualizing, 62
secret panel controller, 3–5, 22, 32–33, 54, 131, 133, 163–164, 438–444, 488, 491
in ANTLR, 549–552
in Java, 530, 535–538, 541–546, 549–552
lexing, 241–244
nonautonomous statements in, 211–218
programming, 9–16
testing, 56–57
using with:
Embedded Translation, 300–304
Model-Aware Generation, 557–563
State Machine, 530
Templated Generation, 541–546
Transformer Generation, 535–538
Tree Construction, 284–291
security systems, 3–5
Semantic Model, 16–21, 22, 25, 29, 42–47, 51–54, 56–57, 60–62, 71–72, 81–83, 92–94, 100, 102, 108, 110–112, 115, 120–123, 130–131, 133, 135–136, 138, 141, 148–149, 152, 159–164, 168, 170, 172, 176–178, 202, 221–223, 282, 284, 288, 299, 301, 320, 345, 349, 353–354, 360, 387, 441, 446, 456, 458–461, 463, 465, 470, 488, 509, 520, 530, 533–534, 556, 563–564, 567, 574, 576
advantages of, 71, 162–164, 306
and code generation, 21, 128, 130, 162, 533–538
and Foreign Code, 310–315, 318
binding XML elements to, 102
executing, 162
fluent constructs in, 344–345
generating diagrams from, 24
handling semantic errors in, 63–64
model-based migration of, 65
populating, 36, 43–47, 49, 52, 128, 162, 284, 288–291, 317, 575
in an invalid state, 60
in Recursive Descent Parser, 248
in Ruby, 35
via a command-query interface, 160
via Embedded Translation, 299–304
schema of, 130
separating from Expression Builder, 71, 344
simulacrum of, 556–557
supporting multiple DSLs for, 163
testing, 53–57
using with:
Annotation, 447
Class Symbol Table, 468
Decision Table, 496–502
Dependency Network, 509–511
language workbenches, 142
Model-Aware Generation, 555–566
Production Rule System, 519–521
Symbol Table, 166
Templated Generation, 125, 544, 546, 549
Transformer Generation, 124, 533–538
validation checks in, 161, 164
visualizing, 34
vs. Domain Model, 44, 134, 160
vs. syntax tree, 48, 135, 160, 306
semantic predicates, 226–227, 271
semantics, 49
Sendmail, 110
separation of concerns, 71
separators, in lexers, 320, 324–325, 333–336
sequence operator
actions for, 259–260
implementing, 257
in Recursive Descent Parser, 246
serialized data structures, 30
servers, 448
setf macro (Lisp), 191
shifting, 98
Simonyi, Charles, 140
SLR parser, 99
Smalltalk
adding methods to third-party classes in, 482
conditional expressions in, 405
Dynamic Reception in, 86
named parameters in, 78
overriding unknown methods in, 428
symbols in, 166
source code repository, 126, 275
source-based editing, 136
Specification, 206, 313, 366, 389
spreadsheets, 23–24, 138–140, 496–497
from C# conditions, 457
mapping to Java classes, 151–152
Starbucks, 31
start conditions, 322–324
State Machine, 4–9, 18, 118, 122, 159–161, 163–164, 527–530
creating, 295–296
dynamically loading, 564–566
lexical analyzer for, 241–244
parsing, 215–218
using with Adaptive Model, 529
visualizing as a diagram, 136
state tables, 124
statement separators, 286, 333–336
states, 528
static data, 352
static import, 73, 352, 360, 371, 378
static methods. See methods, static
and external DSLs, 110
code generation for, 566
compilation errors in, 427
Decision Table in, 496
enums in, 419
Foreign Code in, 100
IDE support for, 377
marker interfaces in, 447
no Dynamic Reception in, 429
string arguments in, 521
type-aware autocompletion in, 467, 469
using Textual Polishing in, 477
statistics, 30
Strategy, 212
strings
and performance, 166
autocompletion of, in IDE, 168
operations with, 166
splitting, 90
strtok function (C), 565–566
structs, 28
structural constraints, 135
subroutines. See functions
sugar. See syntactic sugar
Swiby, 154
symbol data type, 78, 166, 419, 467
Symbol Table, 16, 50–52, 71, 78, 93, 97, 165–174, 215, 218, 259, 289, 441, 468–469, 484, 550
in internal DSLs, 170–171
populating, 442
uses of, 168
using with Semantic Model, 166
values of, 166
Symbol Table for Nested Scopes, 167
symbolic constants, 185
and performance, 166
declaring, 167
dynamically typed, 419
misspelled, 167
operations with, 166
statically typed, 168, 172–174, 467
syntactic analysis, 223–227
in ANTLR, 274–276
separating from lexing, 94–95
strategy of, 89–92
syntactic indentation, 95, 337–339, 478
syntactic noise, 75, 81, 108, 376, 381, 386, 392, 415
of DSLs, 82
of embedded code, 101
reducing with:
Dynamic Reception, 428, 439, 443–444
syntax coloring, 86
syntactic sugar, 358, 360, 437, 443
syntax highlighting, 22, 86, 107, 549
syntax tree, 47, 49, 93–94, 226, 376
abstract. See AST
constructing. See Tree Construction, 260
populating in Recursive Descent Parser, 248
vs. Semantic Model, 48, 135, 160, 306
syntax-directed editing, 22
Syntax-Directed Translation, 49, 79, 90–92, 94, 97, 105–107, 148, 152, 203–205, 219–227, 229–230, 232, 240, 281, 299, 339
composing DSLs with, 111
disadvantages of, 92, 227, 271
grammar files for, 126
learning cost of, 105–106
separation between lexing and parsing in, 221–223, 240–241
strategies to produce output with, 202, 276
syntactic indentation in, 339
using with:
Alternative Tokenization, 326
Foreign Code, 126
Newline Separators, 333–336
vs. Delimiter-Directed Translation, 92, 106, 204–205, 227
vs. Embedded Interpretation, 276
vs. Embedded Translation, 276
vs. Tree Construction, 276
T
tabs, 338
Templated Generation, 183, 192, 539–546
problems with, 125
using with:
Embedment Helper, 125, 540, 544, 548
Ruby, 574
Semantic Model, 125, 544, 546, 549
Transformer Generation, 534
vs. Transformer Generation, 124–125, 540
templating engine, 540
templating languages, 540
terminal symbols
actions for, 259–260
combinators for, 263
recognizer for, 257–259
Test Utility Method, 56
test-driven development, 53
testing, 53–62
marking methods for, 445, 447–448
with invalid input tests, 59–61
Textual Polishing, 85, 185, 477–480
in Ruby, 478–480
using with Dynamic Reception, 428
third-party libraries
adding methods to, 482
wrapping with DSLs, 33
tokenizers. See lexers
checking, 363–365
declaring in grammar file, 287
ignoring type of, 325–326
in ANTLR, 274
nested, 318
type mutation of, 324–325
train wrecks, 68
Transformer Generation, 533–538, 541
input-driven vs. output-driven, 534
using with:
Model-Aware Generation, 535–538
Templated Generation, 534
vs. Templated Generation, 124–125, 540
Tree Construction, 49, 93–94, 101, 152, 168, 225–226, 236–237, 253, 281–297, 300–301, 326, 331
actions in, 259
advantages of, 93
and Parser Generator, 271
code actions in, 292–297
in Recursive Descent Parser, 248
memory consumption of, 284
parsing, 286–288
populating Semantic Model with, 284
processing AST in multiple passes, 302
tokenizing, 285–286
vs. Delimiter-Directed Translation, 202, 286
vs. Embedded Translation, 93, 284, 300
vs. Syntax-Directed Translation, 276
type checking, 87–88
type transmogrification, 375, 481–482
U
Unix
configuration files for, 102
line ending character in, 202
little languages in, 28
up-to operator, 232
V
annotations for, 448–454
in C#, 517–521
on form fields, 448–449
rules for, 516–521
Value Object, 375
Vanderburg, Glenn, 112
varargs parameters, 77, 358, 361, 417–419, 421
in Ruby, 421
variable capture, 186, 190, 192
variables
context. See Context Variable
declaring, 97
limited-scope, 80–81
static, 558
VBA (Visual Basic for Applications), 15
Velocity, 184, 187, 541–546, 552–554
template commands in, 544
version control systems, 35
visualization, 24–25
of Adaptive Model, 491
of scripts, 62
of Semantic Model, 34
of State Machine, 136
Vlissides, John, 571
Voelter, Marcus, 128
W
warnings, 194
web pages
common elements on, 185
embedding code into, 547
generating:
by Embedment Helper, 552–554
in Transformer Generation, 534
in Velocity, 187
Weirich, Jim, 415
whitespace
in Delimiter-Directed Translation, 204
in lexers, 222–223, 241, 286, 319, 325
in parsers, 202
in syntactic indentation, 338
syntactic, 95
WPF (Windows Presentation Framework), 152
X
XML (Extensible Markup Language), 11–12
advantages of, 102
as serialization mechanism, 102
binding to Semantic Model, 102
creating documents in, 415
parsers for, 102
processing, 93
syntactic noisiness of, 13, 101, 103
using for Context Variables, 176
vs. custom syntax, 13
XML Schema, 102
XPath, 83
XSLT (Extensible Stylesheet Language Transformations), 30
Xtext, 141
xUnit, 447–448
Y
YAML (YAML Ain’t Markup Language), 103
indentation in, 338
using for Context Variables, 176
Yoder, Joe, 115