Index
Note: Page numbers followed by ‘f’ indicate figures, ‘t’ indicates tables.
A
Abowd, Gregory, 73
ABS. See Activity Based Search
Abstract knowledge representation, 316
Abstract task model, 315, 316f, 317f
Abstractness, 13
AC. See Accessibility Commons
Access, 411
Accessibility. See also Social Accessibility; Web accessibility
assessment/visualization of, 355–356
database integration and, 352–353
fully-automated approaches to, 352
manual approaches to, 351–352
metadata, 349
of RIAs, 351–352
visualizations of, 355–356
Accessibility Commons (AC)
automation-based indexing and, 359
broken/conflicting metadata in, 363
database performance of, 363
metadata representation in, 358–361, 363
uniting disparate metadata in, 359–361
Accessible Rich Internet Applications (ARIA), 349, 351–352
collaborative crawling and, 356–358
screen readers supported by, 357
ACE. See Attempto Controlled English
ACGT, 275
ACGT OncoSimulator, 275, 277, 279, 282, 284–285
Action history, 379
ActionScript, 290
Active wiki, 192
ActiveX, 407–408
Activity Based Search (ABS), 75
Adeo, 109
Adobe Developer Network, 425
Adobe Flash/Flex, 34–35
Adobe Flex, 425
Air Canada, 376–377
AJAX. See Asynchronous JavaScript and XML
Algorithms
diff, 352
optimizing, 304
recursive, 302–303
sloppy programming and, 300–304
sloppy Web command line and, 301–304
Alternative text metadata, 349
Amazon.com, 29, 32–33, 51, 115, 191, 251
collaborative crawling of, 357–358, 357f
mobile application, 114f
using TrailBlazer with, 373f
American Airlines, 105–106, 106f, 122, 122t
Amtrak, 377
AnalogySpace, 335
Anaphora resolution, 337
Annotation, 328
d.mix Web pages, 197–198
manual, 351
of metadata, 351
Annotation and Retrieval Integration Agent (ARIA), 75
AOL, 51
Apache, 441
APIs. See Application Programming Interfaces
Apple Data Detectors, 72, 73–74, 73f, 82–83
Apple Macintosh, 19
Apple’s Automator system, 208
Application Programming Interfaces (APIs), 25, 191, 290, 291
Atomate and, 188
Flickr, 390
Google Maps, 390, 391, 392–393
Google Search AJAX API, 241, 246
Mash Maker wrappers for, 183
monitoring and controlling physical spaces, 201
sloppy programming and, 291
of social networks, 392–393
Applications, open access to, 17–19. See also specific application
ARIA. See Accessible Rich Internet Applications; Annotation and Retrieval Integration Agent
ASP, 394
Assieme systems, 208–209
Asynchronous JavaScript and XML (AJAX), 40, 58, 118–119, 123, 209, 275–276, 348, 424
API of Google Search, 241, 246
crawling, 352
transcoding with, 350
Athena Blanche, 57–58
Atom Garage, 146
APIs and, 188
Atom Garage, 146
community sharing, 146
data retrieval, 134
delegation interface, 137
entity explorer interface, 137–138, 138f
evaluation, 141–145
expressing actions to be taken, 137
extending the system, 132, 139–141
implementing, 139
life stream publishing, 146
new behaviors created by, 129, 129f, 130f
new data added to, 132, 133 f, 140
ongoing work, 145–147
peRSSona, 146
properties automatically updated, 141
PubSubHubbub, 146–147
pull to push, 146–147
related work, 133–134
rule creation and management interface, 138–139
rule creation study, 142f, 142, 143, 144f, 145f
rule language extension, 146
rules, 135–137
schema extension in, 140–141
simulation and by-demonstration UI, 146
situational constraints, 136
social coordination, 131–132
social networking sites and, 128
specifying conditions for behavior execution, 136
structure editor, 14f
time constraints, 136
triggering and scheduling, 137
user interface, 137–139
walk-through, 128–132
Web mashups inspiring, 134
world model updates, 134, 135, 135f
ATOM/RSS feeds, 146–147
AtomSmasher, 328
Attempto Controlled English (ACE), 133
Attention Investment model, 398, 399f
Authoring cards, 244–246, 245f, 247–248
Autocomplete, 293
Automate
predicates and actions added to, 139–140
simple contextual reminding, 129–131
Automated Click for Charity, 32
Automatic form fill-in, 29
Automation
-based indexing, 359
in Flyspray, 57
Automator system. See Apple’s Automator system
AutoPagerize, 31
AxsJAX, 350, 352, 360–361, 370, 376
B
Babel, 140
Barnes and Noble, 29
Bauer, Mathias, 69
BBC, 257
BEA, 411
Bespin. See Mozilla Bespin
Bioinformatics, 276
BitTorrent, 27
Black box reuse, 411–412, 411t
Bookmarks, 165. See also Smart bookmarks; Social bookmarking systems
Bookstore/library integration, 54–55, 55f
Bragg, William Henry, 280f
Braille, refreshable, 367
Broken wrapper, 180
Broken/conflicting metadata, 363
Browser Helper Objects, for IE, 26, 32
Bug tracking
Flyspray, 32, 55, 55f, 56, 56f
Burnett, Margaret, 12
Buttons, 25
Xerox Lisp, 208
C
C#, 408
C++, 424
C syntax, 290
C3W. See Clip, Connect, and Clone for Web; Clip, connect, and clone for Web
CALO. See Cognitive Assistant that Learns and Organizes
CAPTCHA, 32–33
Carlis, John, 69
Cascading Style Sheets (CSS), 39–40, 200
Web Summaries templates and, 244
WebPageDump and, 253–254
CCrawl, 357
CD. See Cognitive Dimensions of Notations framework
CHCS. See Composite Health Care System
Chickenfoot, 19, 24, 26, 29, 31–32, 31f, 34, 59–60, 63, 86, 181–182, 187, 207, 291, 294
applications, 53–58
bookstore/library integration, 54–55, 55f
clipping in, 44
concatenating sequence of pages, 54, 54f
customizations transforming in, 53, 58
development environment of, 48–49
form manipulation, 45
future work, 60
Google Image Search, 41f
Highlighting vocabulary words in, 53, 53f
JavaScript writing, 42
keyword matching algorithm, 50–51
language of, 42
navigation and page loading, 46–47
other input, 46
page modification, 47
pattern matching, 42–44
PLOW v., 328
related work, 60–62
rewriting Web with, 39–64
security, 49–50
sorting tables, 53–54
synchronous, page-independent nature of, 59
transforming customization in, 53, 58
vocabulary-builder for, 31, 31f
for Web testing, 59
webmail faces, 57–58
widgets, 48
Chickenwire, 29
Cisco Unified Callmanager user options menu, 5f
CiteSeer, 274f
CitingPapers recipe, 274f
Clarifying knowledge, 445–446
grammar-based interpreter, 90, 90f
keyword-based interpreter, 89–90
Clip, Connect, and Clone for Web (C3W), 13, 108–109, 153–172, 168–171, 172, 184, 209, 236, 269, 270–272
automating access to Web applications, 165–167
capturing dependency relationships between elements, 161–162
clipping, 155, 156f, 157f, 158, 159f, 160–163, 168
cloning, 157–158, 160f, 164–165, 164f, 168, 169
connecting, 155–157, 163–164, 168
examples and explanations, 154–158, 155f
extraction, 166–167
formalized extraction of content, 166–167
handling multiple scenarios in parallel, 167–168
implementation requirements, 158–165
passing information between applications, 167
recording and replaying Web
interaction, 165–166
related work, 165–168
replaying derivations, 163
WWW, 153–172
Clipping, 155, 156f, 157f, 158, 159f, 160–163, 168, 372–374
in Chickenfoot, 44
TrailBlazer syntax for, 372–374
CLOnE, 133
Cloning. See also Clip, Connect, and Clone for Web
code, 412
Clustering visualizations, 263, 264f
CNL. See Controlled natural languages
CNLI. See Constrained-input natural language interface; Controlled natural language interface
CNN, 154, 155, 174–175, 175f, 402
COACH, 370
Cobol, 341
Code cloning, 412
Code invocation, 411–412
Code reuse
assumptions in, 409–411
authorship-level evidence of, 415t, 417
code cloning, 412
code invocation, 411–412, 411t
code-level evidence of, 414–417, 415t
conceptual, 412
in end user programming, 407–408
flexibility/composability and, 416
forms of, 411–412
functional size and, 417
future directions in, 419–421
mass appeal/functional relevance and, 416
messiness in, 407
predicting, 418–419
prior use-level evidence of, 415t, 417–418
template, 412
understandability and, 416–417
untapped evidence of, 418
vision v. reality in, 408–411
white box, 412
Cognitive Assistant that Learns and Organizes (CALO), 225, 326–328
Cognitive Dimensions of Notations framework (CD), 398
Cognitive load, sloppy programing and, 293
Cognitive overload, 278–279
Collaborative authoring, 349
Collaborative crawling, 356–358
ARIA and, 356–358
SA and, 356–358
Collaborative debugging
social, 429–437
Yahoo! Pipes and, 430–431, 435f, 436
Collaborative problem solving (CPS), with PLOW, 313–315
Collaborative scripting, for WWW, 85–104
Collapse-to-Zoom, 109
COM objects, 408
Command line interface, sloppy programming and, 294–296, 294f, 301–304, 305
Comments, CoScripter and, 372
Commons-based peer production, 389
Commonsense
leveraging, to understand context, 74–76
Comparative Web Browser (CWB), 167
Complexity ceiling, 203
Composite Health Care System (CHCS), 313
Computer literacy, 3
Concatenating, sequence of pages, 54, 54f
ConceptNet, 66, 74, 75, 82, 83, 328, 337
Conceptual reuse, 412
Conditionals, in PLOW, 322
Conflicting metadata, 363
Connecting, 155–157, 163–164, 168
Consistency in heterogenous environments, 229
Constrained-input natural language interface (CNLI), 127
Consumer Reports, 418
Content-based extraction rules, 239
Context Toolkit, 133
Control constructs, for PLOW, 322–324
Controlled natural language interface (CNLI), 128
Controlled natural languages (CNL), 133
Conventional programming languages, 12
Cookies, 187
Coordinated manipulation, 275–277, 276f
Copy and paste, 174. See also Clip, Connect, and Clone for Web
Intel Mash Maker metaphor, 188
in Tailor, 221
Correct widget, 177
CoScripter, 19, 24, 26, 29, 31–32, 34, 35, 61, 85–86, 103, 104, 170, 180, 207, 209, 276, 300, 304–305, 368f, 410–411, 420, 421. See also ClearScript; Interviews, CoScripter users; Real-world usage study, CoScripter
action types in, 377
automation limitations and issues, 100–101
comments in, 372
debugging and mixed-initiative interaction, 91–92
evaluating effectiveness of, 92
general issues and future work, 102–103
Highlight integration with, 119
human-readable labels, 90–91
limitations and issues sharing, 101–102
method, 99
new processes learning, 86, 95f
participants, 98–99
personal repositories, 95, 96f
PLOW and, 328
procedure sharing and, 229
related work, 86
results, 99–102
scripting language, 89–90
sharing how-to knowledge, 101–102
sidebar, 88f
Slop, 116
sloppy programming and, 296–297, 297f, 300–301, 370
user needs understanding study, 92–97, 95f
variables and personal database, 91
CPS. See Collaborative problem solving
step definition request by, 317
task goal definition for, 316–317
CPSA. See CPS agent
craigslist, 27
Creo, 32, 35, 65, 67, 69–71, 70f, 76, 84, 241, 328
evaluation limits of, 80–82, 81f
generalizing information, 77
implementation of, 76–78
limitations of, 78
recording actions on Web, 77
software ability to improve user efficiency, 79–80
usability of, 78–79
Cross-application scripting, 19
Cross-domain dataflow procedures, learning, 217–218
Cross-domain EUP, 214
open questions in, 228–229
Crowdsourcing, 198–199
csh, 290
CSS. See Cascading Style Sheets
Customizations, 23–24, 32, 40. See also Web customization
desktop Web, 24–25
in Flyspray, 57
generic Web, 30–31
in Mozilla Firefox, 26–33
one-shot Web, 31–32
repeated use Web, 31–32
simplification, 29
site-specific Web, 30–31
transforming, in Chickenfoot, 53, 58
transforming Flyspray bug tracker, 57
CWB. See Comparative Web Browser
CyberDesk, 73–74
D
DaniWeb, 390
Dapper.net, 183
Data detectors, 71–74. See also Google AutoLink; Microsoft Smart Tags; Miro
goal-oriented Web browsers, 71–74
research on, 72–73
Data directors, 19–20
Data integration, 242–243
Data Mapper. See DAPPER
Data mining, 19–20
Data retrieval. See also Parallel retrieval Atomate, 134
Data sources, extending to new, 132
Data tree
showing additions from widgets, 184f
Database
AC performance, 363
components, 408
coScripter variables and personal, 91
integration, 352–353
integration and SA, 352–353
metadata in Accessibility Commons, 358–361, 363
Dataflow
procedures, 216, 216f, 217–218
supports in LAPDOG, 216, 216f, 217
DBLP, 274f
DBpedia, 280–281, 280f, 282–284
classical models of, 424–425
CoScripter, 91–92
guidelines for, 424–425
mental models and, 425–426
social collaborative, 429–437, 435f
in software development, 423–424
tools/strategies in, 425
Web use for, 445–446
WWW and, 445–446
Yahoo! Pipes and, 430–431, 435f, 436
Declarative language, 21
Delegate to Remember the Milk, 29
Delegation interface, 137
del.icio.us, 86
Delta, 376–377
Design review study, Atomate, 141, 143
Desktop Web customization, 24–25
DEVise, 255
Dey, Anind, 73
DHTML. See Dynamic HTML
Dice coefficient, 258–259
Diff algorithm, 352
Digestor, 108
Digg, 389
d.mix, 109, 183, 191–193, 210, 211
additional applications, 201–203
cluttered Web pages and, 201
deploying applications outside wiki environment, 207
evaluation, 203–207
finding and appropriating documentation and code, 208–209
foraging for examples, 193
implementation, 195–200
inconsistent models for sampling, 206
integrating site and service, 192, 192f
lack of documentation and error handling, 206–207
limitations and future work, 209–210
multi-language scripting, 206
parametric copy, 200
program by sample, 193–195
related work, 207–209
sample this button, 195–197
shortcomings, 206–207
site-to-service map authoring and maintenance, 198–199, 199f
study protocol, 204
tools for end user modification of Web experiences, 207
tools for end user synthesis of Web experiences, 208
virtually editing existing Web pages, 201
Web applications to monitor and control physical spaces, 201–203, 202f
Web pages annotation, 197–198
Web professional feedback, 203
wiki, 195
as wiki host and script executer, 199–200
Document Object Model (DOM), 23–24, 25, 198–199, 207, 209, 313
Web page structure of, 320f
Web Summaries and, 238–239, 240
WebPageDump and, 253–254
Dojo Dijit, 352
DOM. See Document Object Model
DOM inspector, PLOW in, 318–319, 318f
DOM mutation event listeners, 352
DOM tree, 276
Dontacheva’s Relations, Card, and Search Templates system, 117
Dr. Dobb’s Journal, 418
Drop Zones, 183
Dynamic HTML (DHTML), 244
Dynamic script generation, in TrailBlazer, 376–381
E
ebay Sidebar, 32
Elastic Windows, 273
E-learning, 349
Eli Lilly, 57
Emacspeak, 369
End user programming (EUP), 213
approaches to, 12–17
code reuse in, 407–408
control of data and services, 76
cross-application scripting, 19
evidence of reusable code in, 413–418, 415t
forms of reuse in, 411–412, 411t
gray areas, 8
HTML, 20–22
mobile, 22
origins of, 3
PBD system, 13–17
programming-by-example systems and, 73–74
reuse in, 407–408
scripting languages, 12–13
semantic information, 19–20
social networks, 19
special case of spreadsheet programming, 11–12
Web removing barriers to, 17–22
what is, 4–12
gather hotel information, 6, 9f
nutritional information additions, 6–8, 10f
programming desire of, 4–8
text alerts, 5–6
Entity explorer interface, 137–138, 138f
ESP game, 348
EUP. See End user programming
Excel. See Microsoft Excel
Execution failure, in PLOW tasks, 325–326, 327f
EXPECT, 341
Expedia, 176
Explanatory power, 303–304
Exploratory Web research, 235–236
External memory, 437, 440, 446
for syntax, 437
WWW as aid, 446
External metadata, 349–350
Extraction
in C3W, 166–167
content-based rules, 239
patterns of, 236–237, 239–240, 239f, 247–248
rules, 238–239
temporal, 256
F
Faaborg, Alexander, 76–77
Facebook, 134, 137, 348, 389, 392–393, 402, 403–404
Client-Connect, 139
mashup showing friends, 185f, 186
Federal Express, 377
Felicity Conditions, 341
Filters, on lenses, 259–260
Fire Vox, 360–361
Firefox. See Mozilla Firefox
Firefox extensions. See specific extension
Flash, 34–35
Flashblock, 29
Flickr, 27, 29, 30, 30f, 191, 193, 195, 195f, 196f, 197, 198, 204, 206
API of, 390
Flyspray bug tracker, 32, 55, 55f, 56, 56f
automation in, 57
transforming customization in, 57
Form elements labels, 349
Form manipulation, in Chickenfoot, 45
Formatting constraints, 243–244
Forms/3, 167
Fortran, 411–412
Frame based parsing, 335
FRAMENET, 335
Freebase.com, 183–184
FreshDirect, 69
Froggy, 31
G
GData JS, 139
Generic Web customization, 30–31
GINSENG, 133
gist.github.com, 430
Globefish, 31
Gmail, 27, 30, 33, 34, 360–361
GMiF, 27, 29, 30f. See also Flickr; Google Maps
Goal-oriented Search Engine (GOOSE), 75
A Goal-Oriented User Interface for Personalized Semantic Search (Faaborg), 76–77
Goal-oriented Web browsers, 65–84, 67, 68f
ConceptNet, 66, 74, 75, 82, 83, 328, 337
data detectors, 71–74
end users in control of data and services, 76
future work, 82
semantics, 74
Stanford TAP, 67, 74, 82, 83, 328
teaching computers, 66–67
Google, 27, 33, 51, 352, 389. See also Gmail; iGoogle
goal of using, 372
parallel retrieval for, 273
query suggestions in, 305
Google Account Multi-Logon, 27
Google AutoLink, 65–66, 73, 74, 75, 76
Google Calendar, 29
Google Docs, 360–361
Google Image Labeler, 349
Google Mail, 40
Google Maps, 23–24, 25, 27, 29, 30f, 40, 175, 187, 191, 209–210, 275–276
Google Notebook, 430
Google Search AJAX API, 241, 246
GoogleAutoPager, 31
GOOSE. See Goal-oriented Search Engine
GPS Observations, 141
Grammar-based interpreter, 90, 90f
human-readable labels, 90–91
Grammars by Example. See Grammex
Grammex, 74
Graphical user interface (GUI), 293, 294f
Greasemonkey, 24, 26, 27–29, 32, 33, 34, 41–42, 61, 181–182, 187, 207, 360–361. See also LookItUp Greasemonkey script; Virtual Keyboard Interface Greasemonkey
GUI. See Graphical user interface
H
HANDS system, 290
Heterogenous environments, 229
Heuristics
in ITL, 221
in LAPDOG, 217
in PLOW, 319
sloppy programming and, 305
Highlight, 32, 105–126, 105–108, 106f, 125, 126, 370
architecture of, 117–119, 118f
background, 108–109
breadth and benefits, 121–122, 122t
content operations, 113–115
CoScripter integration with, 119
discussion, 123–124
flight tracking application, 105–106, 106f
informal user study, 120–121, 120f
interface walkthrough, 110–113, 111f, 112f
pagelets and transition events generalized, 116–117
system, 109–119
Web page elements identified, 115–116
Highlight Designer extension, 109–113, 111f, 112f, 114f
XPath expressions, 116
Holiday Inn example, 16f
How-to knowledge, recording/playback of, 370
HTML, 20–22, 25, 26, 34–35, 200, 209, 394. See also Dynamic HTML
Chickenfoot related to, 40
EUP, 20–22
Human-readable labels, 90–91
Hunter Gatherer, 165, 209, 236
I
IBM corporation study, 97
email survey of lapsed users, 98
recording user activity, 97–98
user interviews, 98–102
IBM QEDWiki, 208
iCAP, 133
IDE. See Integrated development environment
IE. See Internet Explorer
iMacros, 86
IMDb. See Internet Movie Database
Indirectness, 13
Inferring intent, 13–15
InfoBox, 167
Informal learning, 390
Informal user study, Highlight, 120–121, 120f
commands in, 298–299
feedback in, 299–300
sloppy programming and, 298–300
Integrated development environment (IDE), 292
Integrated Task Learning (ITL), 213, 214, 230, 231. See also LAPDOG; primTL; Tailor
action-oriented domain model for, 225–226
conditions/loops in, 222, 223f
copying procedures in, 221
cross-domain dataflow procedure learning in, 217–218
editing errors/flaws in, 223
extensible type system in, 227–228
framework, 420
heuristics in, 221
interface of, 217f
mapping actions across domains, 224–225
modeling human-level actions in, 226–227
modeling objects/relations in, 227
pluggable domain modules for, 225–228
procedure learning in, 215–218
procedure visualization/editing in, 218–223
typed dataflow editing in, 220–221
visualization templates in, 219
Intel Mash Maker, 32, 62, 170, 173–190, 188, 189, 420, 428
browsing behavior, 179
browsing Web example, 174–175
copy and paste, 188
editing wrapper for Yelp, 182–183
flight booking example, 176–177
Mozilla Firefox extension, 173
overlay mashups, 177–178
privacy/accuracy trade-off, 179
shared data tree, 184–186, 184f
Web wrapper collaborative creation, 179–182
widget implementation, 187
wrapper editor, 183–184
Intel Selection Recognition Agent, 72, 73–74, 79
Intel Software Solutions Group, 173, 174, 188
Interclue extension, 30
Interleaved demonstration/editing, 228
Internal metadata, 349
Internet Explorer (IE), 160, 270–272
Browser Helper Objects for, 26, 32
Mash Maker extension for, 173
SA plug-ins for, 353
Internet Movie Database (IMDb), 27, 34, 79
Interviews, CoScripter users, 98–102
automation limitations and issues, 100–101
limitations and issues sharing, 101–102
method, 99
participants, 98–99
results, 99–102
sharing how-to knowledge, 101–102
iPhone. See Apple iPhone
Irmak/Suel, 183
Irreversibility of actions, 352
Iteration, in PLOW, 322–324, 323f
ITL. See Integrated Task Learning
J
JavaScript, 25, 26, 34–35, 40, 41, 59, 60, 118–119, 123, 139–140, 171, 187, 200, 206, 244, 289–290, 391–392, 425, 426–428
AxsJAX and, 352
Chickenfoot written in, 42
reuse of, 407
transcoding with, 350
WebPageDump and, 253–254
JavaScript Object Notation (JSON), 428–429
JSON. See JavaScript Object Notation Just-in-time learning, 390, 437
key insights for, 424–425
opportunistic programming and, 438
with Web tutorials, 437
K
Kalbag, Sameer, 72
Kalpana, 32
Kapow, 208
Karma, 188
Kay, Alan, 15–17
Keyword matching algorithm
in Chickenfoot, 50–51
Keyword-based interpreter, 89–90
Keywords, 44
Knowledge encoding, 328
Kohl’s, 402
Konstan, Joseph, 69
Koseki, Yoshiyuki, 69
K-Plex, Inc., 158
L
LabView, 12
LambdaMOO, 339
LAPDOG, 215
dataflow supports in, 216, 216f, 217
heuristics in, 217
parameter generalization with, 216–217
structural generalization in, 217
Latency, 352
LaTeX, 214
Lau, Tessa, 290
Layout templates, 243–244, 244f, 247–248
applying filters for, 259–260
filters on, 259–260
inspirations for, 255
magic, 255
stacking, 261–262
structural, 256, 257–258, 258f
types of, 256
LexisNexis, 27
Library/bookstore integration, 54–55, 55f
Lieberman, Henry, 74
Life stream publishing, 146
Linked Data widget, 175
LinkedIn, 348
Live Clipboard, 430
Long Tail, 402
LookItUp Greasemonkey script, 31
M
Machine learning, 310
Macintosh. See Apple Macintosh
Mackay, Wendy, 24–25
Macros. See also iMacros; Web macros
recording systems for, 241
reusing, 407
Magic Lenses, 255
Mag.nolia wishlist, 29
Management interface, in Atomate, 138–139
Manipulation, coordinated, 275–277, 276f
Manual annotations, screen readers and, 351
ManyEyes, 348
Map widget, 186
Mapping
ITL actions across domains, 224–225
semantic, 224–225
Mash Maker. See Intel Mash Maker
Mashup(s), 23, 29, 173, 174f, 175, 184, 389–390. See also Intel Mash Maker; Overlay mashups
Atomate inspired by, 134
clarification of, 177
difficulty ratings of, 396–397
legal concern reduction, 178
misbehavior prevention, 177
motivations for creating, 398–399
predicting future engagement in, 400–402, 400t
problem areas in, 391–392
reasons for creating, 396f
research prototypes and, 426
suggested, 178–179
systems for, 328
underlying service/infrastructure dependence of, 427
usefulness ratings of, 397
users learning, 392
Mashup Awards, 25
MashupOS, 187
MATLAB, 413
reuse of, 407
MDAT system, 108
Mechanical Turk, 348
Meme Media Laboratory, Hokkaido University, 169
debugging and, 425–426
Metadata
accessibility, 349
in Accessibility Commons database, 358–361, 363
AJAX crawling for, 352
alternative text, 349
annotation of, 351
authoring approaches for, 350–351, 356
authoring skills for, 364
broken/conflicting, 363
conflicting, 363
external, 349–350
fully-automated generation of, 350
internal, 349
management tools for, 351
Pagemap for authoring, 356
SA and, 349–350, 353, 358–359, 364
Semantic Web and, 352–353
semi-automated authoring of, 350–351
structural, 348
transcoding with external, 349–350
types of, 349
uniting disparate, 359–361
Web pages in, 349
Metafor, 291
MOOIDE and, 333
Meta-URLs, 293
Mica systems, 208–209
Microformats, 19–20
Microsoft Excel, 11, 160, 410–411
Microsoft Live Labs, 430
Microsoft Outlook, 209
Microsoft Popfly, 184, 185, 186–187, 428
Microsoft Powerpoint, 426
Microsoft Silverlight, 34–35, 171
Microsoft Smart Tags, 65–66, 73, 75, 76
Microsoft Visio, 411
Microsoft Windows, 19
Miller, James, 72
Miller, Rob, 69
Minimo, 121
evaluation limits of, 80–82, 81f
implementation of, 76–78
limitations of, 78
natural language ambiguities, 75–76, 76f
software ability to improve user efficiency, 79–80
usability of, 78–79
MIT412, 29
Mixed-initiative interaction, 309–310, 312
Mixed-initiative learning, 228–229
Mobile end user programming, 22
Mobile Web customization, 35
Modern end user, 19
Money Magazine, 418
Montebillo Splendid Hotel, Florence, 9f
MOO, 341
Web-based, 332–333
Commonsense reasoning in, 336–337, 341
dialog manager in, 336–337
interaction with, 334–335
Metafor and, 333
NLP in, 335
parsing in, 335
work related to, 341
MooseCrossing, 290
Mozilla, 76
Mozilla Bespin, 200
Mozilla Firefox
Atomate implemented in, 139
Highlight Designer extension in, 109–110, 114f
Mash Maker extension for, 173
PLOW in, 318
Supporter Tool plug-in for, 354, 355–356, 355f, 356f
transcoding and, 350
Web customization examples, 26–33
Web Summaries extension for, 236–237
Zoetrope and, 253–254
Mozilla Weave, 35
Mozilla Website, 26
Multiple interpretations, 15
Multiscenario Web access, with RecipeSheet, 269–273
My stuff, 137–138. See also Entity explorer interface
My Yahoo, 23–24
Myers, Brad, 69
MySpace, 348
MySQL, 441
N
Naive Bayes classifier, 379
Natural language (NL), 309, 310, 328, 336–337. See also Controlled natural languages; Pseudonatural language
output parameter identification and, 321
programming intent of, 338
Natural language processing (NLP), 290
in MOOIDE, 335
Stanford link parser for, 335
Natural programming, 290, 291, 341
Navigation, in Chickenfoot, 46–47
.NET, 408
New York Times, 57
NL. See Natural language
NLP. See Natural language processing
Norman, Don, 12
Noun phrase identification, 321f
Now Web, 251
nwsource.com, 244–245
O
Object hierarchies, 337
Object relation mapper (ORM), 139
Obscure syntax, 13
Observation-based learning, 328
OMCS. See Open Mind Common Sense
on Cue, 73
OncoRecipeSheet, 283f
One-at-a-time interfaces, 267–268
One-shot Web customization, 31–32
Ontology of objects, 337
OPA Browser, 109
Open Mind Common Sense (OMCS), 66, 67, 75, 337, 337f
Open Object Rexx, 269
Open source movement, 17–19
Opera, 26
Opera Mobile, 121
Opportunistic programming, 425–426, 437
future work in, 450–451
just-in-time learning and, 438
lab study of, 425–426
study method for, 426–428
study results of, 428–429
Web search log analysis for, 446–450
Web tutorials and, 444
Orbitz, 370
OreO, 177
ORM. See Object relation mapper
Output panel, 48
Overlay mashups, 173, 174, 177–178
OWL. See Web Ontology Language
P
Pagelets, 116–117
in Highlight, 116–117
repetitive page structure, 116
repetitive site structure, 116
Pagemap, 356
PageTailor, 109
Palm Pre, 105
Pandit, Milind, 72
Papert, Seymour, 341
The Paradox of Choice (Schwartz), 278–279
Parallel retrieval, 273–275, 274f
for Google, 273
using RecipeSheet, 273–275
for Web-based information-seeking tasks, 273–275
Parameter generalization, 216–217, 328
Parametric copy, 200
Parsing
frame based, 335
limitations in, 339
in MOOIDE, 335
syntactic, 335
Paste widget, 177
Pastebin, 430–431
pastie.org, 430
Path query, 136
Pattern detection, 74
Pattern matching, 42–44
Patterns panel, 48
PBD. See Programming by demonstration
Personal information management (PIM) tools, 127, 128, 138
Web capabilities of, 128
Personal information management tools (PIM), 27
peRSSona, 146
Pew Internet and American Life Project, 393
PHP, 206, 391–392, 394, 441, 426–428
Piccolo, 255
Pictor, 183
Piggybank, 140
PIM. See Personal information management tools
Pinky, 305
Pipes. See Yahoo! Pipes
The Pirate Bay, 32–33
Placement constraints, 243–244
Platypus, 41–42
Play-by-play
high-level step description in, 317
teaching Web tasks with, 315–324
PlexWare platform, 158–159, 160
PLOW. See Procedure Learning On the Web
Popfly. See Microsoft Popfly
PPIG. See Psychology of programming
Primitive actions, 318–319
primTL, 224–225
PrintMonkey, 32
Procedural knowledge
expressing, 370
representing, 370
in TrailBlazer, 370
Procedure Learning On the Web (PLOW), 241, 309, 310–311, 311f, 330
architecture of, 312–315
AtomSmasher v, 328
Chickenfoot v, 328
collaborative problem solving with, 313–315
conditionals in, 322
contextual information in, 310
control constructs for, 322–324
CoScripter and, 328
in DOM inspector, 318–319, 318f
evaluation of, 326–328
execution failure in, 325–326, 327f
heuristics in, 319
improving tasks in, 325–326, 327f
information flow in, 313, 315f
interface for, 313, 314f, 319f
mixed-initiative interaction, 309–310
in Mozilla Firefox, 318
natural language and, 309
parameter identification for, 320–321
persistent/sharable tasks in, 324
proactive execution in, 314–315
proactive interaction with, 310
reusing tasks in, 325
task goal definition in, 316–317
task hierarchy in, 321
task invocation in, 324–325
task management in, 324f
task representation in, 315–316, 316f
task step definition in, 317–319
tRIPS system in, 312–313
utilizing/improving taught tasks in, 324–326
work related to, 328
Procedure sharing, 229
Programming by Demonstration (PBD), 13–17, 193, 309
inferring intent, 13–15
multiple interpretations, 15
presenting programs to user, 15–17
semantic knowledge, 15
Programming by sample, 193–195, 194f, 195f, 202f
Programming-by-example systems, 65–84, 68–71, 83, 193
end user programming and, 73–74
research on, 69
Project Athena, 24–25
Protege, 183–184
Prototype, ideate, and discover, 437
Pseudonatural language, 370
Psychology of programming (PPIG), 425
PubSubHubbub, 146–147
Python, 40, 60–61, 206, 289–290, 333, 334
Q
QEDWiki. See IBM QEDWiki
Qtive, 73
Query formulation, 242
Query refinement, 432f, 437, 449–450
Quicksilver, 298
R
RateMyProfessor, 403
RDF. See Resource Description Framework
Reactive automation, 128
Real-world usage study, CoScripter, 97–98
collaborating over scripts, 98
email survey of lapsed users, 98
log analysis, 97–98
script usage patterns, 97–98
reCAPTCHA, 348
additional scenarios in, 269–270, 271f
CitingPapers recipe, 274f
coordinated manipulation with, 275–277
dependency connections in, 271f
initial evaluations of, 279–283
parallel retrieval using, 273–275
prebuilt recipes in, 270f
tentative composition with, 277–278, 278f
Recombinant Web, 201
Recursive algorithms, 302–303
Refreshable Braille, 367
Regex, 271f
Relevance-feedback retrieval interface, 279
Remember the Milk for Gmail, 32
Reminders, 446
Remote Grid services, 282
Repeated use Web customization, 31–32
Repetitive operations, automating, 33, 40
Resource Description Framework (RDF), 128, 132, 139, 148, 352–353
REST, 269
RIAs. See Rich Internet Applications
Rich Internet Applications (RIAs), 348, 350–351, 356–357. See also Accessible Rich Internet Applications
accessibility of, 351–352
Ruby, 197, 198, 199, 200, 206, 290. See also d.mix
Rule chainer, 134
Rule creation
interface, in Atomate, 138–139
study, of Atomate, 142, 142f, 143, 144f, 145f
Rule firings, 134
Rule language extension, Atomate, 146
RuleML, 133
S
SA. See Social Accessibility
SADIe. See Structural semantics for Accessibility and Device Independence
Safonov, Alex, 69
Same-origin policy, 49–50
Sample this button, 195–197
Sampling, 192
Saxon, 254
Schwartz, Barry, 278–279
Scraping. See Web scraping
Screen readers, 348, 350–351, 367
ARIA support in, 357
augmenting, 369–370
manual annotations and, 351
Screengrab!, 253–254
Scripting
collaborative, 85–104
cross-application, 19
d.mix multi-language, 206
social, 86
Scripting languages, 12–13
CoScripter, 89–90
end user programming, 12–13
interface, 289–290
syntax of, 12–13
of WWW, 40
Search engines. See also specific type
log analysis of, 446–450
synonym searching with, 445
translating terminology with, 437
transparency in, 273
Search result comparison, 242–243
Search template, 246–248, 247f
Second Life, 332–333
Selenium, 86
Semantic knowledge, 15
Semantic mapping, 224–225
Semantic Web, 133, 147, 171, 236, 281
metadata and, 352–353
knowledge bases of, 328
structural, 369
Set function, 137
sh, 290
Shortcuts, 29
Sidebar
CoScripter, 88f
ebay, 32
Silverlight. See Microsoft Silverlight
Simple contextual reminding, 129–131
Simplification customizations, 29
Simulator validation, 282
Site Pattern Analyzer (SPA), 351
Site-specific Web customization, 30–31
Site-to-service correspondence, 191
Site-to-service map, 195–196, 196f, 198
d.mix authoring and maintenance, 198–199, 199f
Site-wide Annotation, 351, 360
Situational constraints, 136
Slashdot.org, 418
Slop, 116
Sloppy programming, 289–290, 291–293
algorithms in, 300–304
APIs and, 291
benefits of, 292–293
cognitive load and, 293
as command line interface, 294–296, 294f, 301–304, 305
CoScripter and, 296–297, 297f, 300–301, 370
discussion of, 304–305
explanatory power and, 303–304
future work in, 305
heuristics and, 305
Inky and, 298–300
interfaces in, 292
key insight behind, 291–292
limitations of, 304–305
related work to, 290–291
systems for, 293–300
Web command line with, 294–296, 301–304, 305
Sloppy Web command line, 294–296, 294f
algorithm for, 301–304
A Small Matter of Programming, 11
Smart Tags, 76
Smartphones, 22
SMash, 187
Snap, 167
SOAP, 269
Social Accessibility (SA), 347, 348, 365, 366, 369. See also Accessibility Commons
accessibility assessment/visualization with, 355–356
architecture/workflow of framework of, 353–354, 354f
authoring skills and, 364
broken/conflicting metadata and, 363
browser plug-ins for, 353, 354
CCrawl and, 357
challenges/future work for, 362–365
collaborative authoring and, 349
collaborative crawling and, 356–358
content provider collaboration and, 364–365
database integration and, 352–353
database performance issues, 363
incentive effectiveness and, 364
metadata in, 349–350, 353, 358–359, 364
Mozilla Firefox plug-ins, 353, 354
pilot service of, 361–362
security/privacy/spam issues, 363
Supporter Tool of, 354, 355–356, 355f, 356f
transcoding in, 349–350
We-LCoME and, 349
work related to, 348–353
Social bookmarking systems, 86
Social collaborative debugging, 429–437, 435f
Social coordination, 131–132
Social networking sites, 128, 348 See also specific site
APIs of, 392–393
Atomate and, 128
end user programming, 19
Social scripting, 86
Software applications with commonsense, 65–84
ConceptNet, 66, 74, 75, 82, 83, 328, 337
Open Mind Common Sense Project, 66
Stanford TAP, 67, 74, 82, 83, 328
teaching computers, 66–67
Software development
debugging in, 423–424
models of, 424
Web and, 426–428
Solvent, 198–199
Sorting tables, 53–54
SPA. See Site Pattern Analyzer
SPARQL, 280–281
SparTag.us, 32
Speech act categories, 338
Speech recognition, 312
Sphinx-III, 312
Spreadsheet programming, 11–12
SQLite, 139
Stacking lenses, 261–262
Stagecast Creator, 12
Stalker, 183
Stanford TAP, 67, 74, 82, 83, 328
State explosion, 352
Stencils, 86
Stop sampling button, 195–196, 197
Structural generalization, 217
Structural lenses, 256, 257–258, 258f
Structural metadata, 348
Structural semantics for Accessibility and Device Independence (SADIe), 351, 361
Styling information, 351
Subjunctive interfaces, 164, 267–269, 284. See also Cloning
defining, 268
user control through, 269
Sugiura, Atsushi, 69
Summary composition, 243–246
Supporter Tool plug-in, 354, 355–356, 355f, 356f
SurfCanyon, 27
Surprise-Explain-Reward strategy, 12
SWRL, 133
Synonyms, 445
Syntactic parsing, 335
Syntax
C, 290
errors in, 289
obscure, 13
of scripting languages, 12–13
TrailBlazer clipping, 372–374
Web as external memory for, 437
Synthesized voice, 367
T
Tag cloud, 198
Tailor, 221
conditions/loops in, 222, 223f
copying procedures in, 221
editing errors/flaws in, 223
typed dataflow editing in, 220–221
Technology initiative, 404
Templates
code reuse, 412
layout, 243–244, 244f, 247–248
layout, in Web Summaries, 243–244, 244f, 247–248
matching, 351
reuse, 412
visualization, in ITL, 219
Web Summaries, and CSS, 244
Temporal extraction, 256
Tentative composition, 277–278, 278f
Terms of Service agreements, 32–33
Textual lenses, 256, 258–259, 259f
Thumbtack, 430
Time constraints, 136
Time series visualizations, 263
Tiramisu recipe, 10f
T-Mobile G1, 105
Tokenizing input, 301–302
Toolkits, 21
Context, 133
TrailBlazer, 119, 367–369, 368f, 383–385, 386
action history in, 379
Amazon.com with, 373f
automatic clipping script component for, 377
clip command in, 372–374
design guidelines for, 371–372
dynamic script generation for, 376–381
dynamic Web content and, 376
evaluation of suggestions in, 382–384
formative evaluation of, 374–376
future work for, 384
general suggestion script component for, 377, 378–379
goto script component for, 377, 378
leveraging existing scripts for related tasks in, 380–381
likelihood action pair feature of, 380
presenting suggestions in, 381
prior action script similarity feature of, 380, 381
procedural knowledge in, 370
region description study for, 372–373
script creation in, 368
script repository-based suggestions in, 380
suggestions in, 377–378, 379–381
task script similarity feature of, 380, 381
work related to, 369–370
Trainable Information Agents (TrIAs), 69, 166
Transcoding, 349–350
with AJAX, 350
authoring approaches for, 350–351
with external metadata, 349–350
with JavaScript, 350
Mozilla Firefox, 350
in SA, 349–350
Web accessibility for, 349–350
of Web pages, 350
XML and, 360
Transforming customization
in Flyspray, 57
Travian Web-based multiplayer game, 27–29
TravianBeyond, 27–29
TrIAs. See Trainable Information Agents
Triggering and scheduling, in Atomate, 137
Triggers panel, 48
TRIPS system, 312f
in PLOW, 312–313
reference resolution capability of, 321
task description from, 317
Trusted widget, 187
U
Underwriters Laboratories, 418
Unformatted addresses, 14f
Unique identifiers (URLs), 23, 293
United, 376–377
United Rail Line, 377
Unix pipes, 208
UPS, 377
URLs. See Unique identifiers
U.S. Geological Survey (USGS), 263
User control, subjunctive interfaces for, 269
User interface design evaluation, 78–82, 80f
User needs study, CoScripter, 92–97
method, 93
participants, 92–93
processes, 93–94
results, 93–95
sharing knowledge, 94–95
USGS. See U.S. Geological Survey
V
Vaquita, 108
VegeTable, 170
Video Cube, 255
Virtual environments, 332–333
Virtual Keyboard Interface Greasemonkey, 31
Visualizations
of accessibility, 355–356
procedure, editing in ITL, 218–223
templates in ITL, 219
time series, 262–263
timelines/movies for, 262–263
with Zoetrope, 262–264
Vocabulary
-builder for Chickenfoot, 31, 31f
words highlighting, in Chickenfoot, 53, 53f
VoiceXML, 360
W
Weather Channel, 51
Web 2.0, 360–361, 403–404, 405
Web accessibility, 347–348
collaborative authoring and, 349
external metadata for, 349–350
improving, 369–370
transcoding for, 349–350
Web automation. See Chickenfoot
Web customization, 23–24, 25–26
annoyance and, 33
creator of, 32
on desktop, 24–25
examples of, 26–33
generic v. site-specific, 30–31
kind of site, 27–29
mobile, 35
Mozilla Firefox examples of, 26–33
one-shot v. repeated use, 31–32
optimizing work habits with, 33
reasons for, 33–34
redundancy eliminated with, 33
repeated use, 31–32
repetitive tasks and, 33
Site-specific, 30–31
summary and future directions, 34–35
targeted site relationship with, 32–33
types of, 29
uniqueness of, 34
usability problems and, 33–34
Web objects
DOM structure of, 320f
in primitive actions, 318–319
Web Ontology Language (OWL), 352–353
Web pages
applying mashup to, 179–182
d.mix and cluttered, 201, 202f
d.mix virtual editing of, 201
metadata in, 349
transcoding of, 350
Web scraping, 209
Web search
log analysis for opportunistic programming, 446–449
Yahoo!, 204
Web service calls, 191
Web service definition language (WSDL), 392
Web spiders, 348, 350–351, 357
Web Summaries, 32, 62, 237f, 238f, 250
authoring cards in, 244–246, 245f, 247–248
discussion of, 247–248
layout templates in, 243–244, 244f, 247–248
Mozilla Firefox extension, 236–237
predefined layout templates in, 243–244
query formulation for, 242
search result comparison for, 242–243
semiautomatic content gathering with, 238–243, 239f
summary composition in, 243–246
system design of, 237–247
template-based search in, 246–248, 247f
templates and CSS, 244
Web tutorials
just-in-time learning with, 437
opportunistic programming and, 444
Web-active end user, 389–390, 405
defining features of, 404
discussion of, 403–405
features of, 393–402
online activities of, 402–404
technology expertise of, 404–405
technology initiative of, 404
Web technology survey of, 393–397, 394t, 395t, 396f
Web-based information-seeking tasks
classification of, 275
one-at-a-time interfaces for, 267–268
parallel retrieval for, 273–275
WebInSight, 350
WebInSight for Images, 360
Webmail
Chickenfoot adding faces to, 57–58
WebPageDump, 253–254
Websites
integrating multiple, 40
Mozilla, 26
transforming appearance of, 39–40
WebThumb, 109
We-LCoME, 349
White box reuse, 412
in Chickenfoot, 48
correct, 177
Data tree showing additions from, 184f
linked Data, 175
map, 186
Mash Maker implementation, 187
paste, 177
trusted, 187
untrusted, 187
WIEN, 183
Wikipedia, 19, 29, 30, 34, 198–199, 280–281, 349, 389, 423
Wikis, 349. See also IBM QEDWiki; MediaWiki
active, 192
Witten and Mo’s TELS system, 13
Wood, Andrew, 73
WordNet, 66
WordPerfect, 24
World model updates, in Atomate, 134, 135, 135f
World Wide Web (WWW). See also Clip, Connect, and Clone for Web; Semantic Web; Websites
automating access to applications of, 165–167
automating repetitive operations, 33, 40
C3W for, 153–172
Chickenfoot rewriting, 39–64
collaborative scripting for, 85–104
debugging with, 445–446
development, 426–428
d.mix professional feedback, 203
EUP barriers removed by, 17–22
evolution of, 347–348
exploratory research, 235–236
as external memory aid, 446
intentions behind use of, 429–437, 431t
learning from, 82
modern end user of, 19
open access to applications provided by, 17–19
recording and replaying interactions, 165–166
scripting languages of, 40
software development, 426–428
varied uses of, 213–214
wrappers collaborative creation, 179–182
Wrappers, 173, 174, 174f, 175, 184
broken, 180
collaborative creation on WWW, 179–182
editor, 183–184
editor for Mash Maker, 183–184
Mash Maker, for APIs, 183
Web, collaborative creation, 179–182
Yelp editing, 182–183
WSDL. See Web service definition language
W3C, 351–352
WWW. See World Wide Web
X
Xerox Lisp Buttons, 208
XML, 40. See also Asynchronous JavaScript and XML
RecipeSheet and, 269
transcoding with, 360
in Zoetrope, 254
XML parsers, 408
XPath, 43, 44, 116–117, 166, 170, 238–239, 254, 257–258, 360–361
XPCOM. See Firefox XPCOM
XQuery, 269
XSnippet, 291
Y
Yahoo!, 27
Yahoo! Maps, 25
Yahoo! Pipes, 23–24, 62, 140, 147, 184, 185, 186–187, 208, 424, 428–429, 429f, 430–431
abstraction layers of, 434–436
collaborative debugging and, 430–431, 435f, 436
data types in, 428
developer forums of, 431, 431t, 432, 432f, 433–434, 433f
visual editor of, 429f, 436, 437f
Yahoo! Search Monkey, 181
Yahoo! Toolbar, 32
Yahoo! Web search, 204
YeastHub, 353
wrapper editing for, 182–183
YouTube, 19, 27, 28f, 29, 30, 34, 191, 195, 198, 204, 389
Z
architecture of, 253–255, 253f
crawler in, 253–254
engine of, 254–255
filtering lenses in, 259–260
interface of, 255
Mozilla Firefox and, 253–254
overview of, 253–254
stacking lenses in, 261–262
storage/data in, 254
time series in, 263
timelines/movies in, 262–263, 263f
visualizations with, 262–264
XML in, 254