Index
Please note that index links point to page beginnings from the print edition. Locations are approximate in e-readers, and you may need to page down one or more times after clicking a link to get to the indexed material.
A
abstraction
avoiding overengineering, 40–41
promoting simplicity, 38–40
reducing coupling on higher levels of, 44–46
access patterns
data modeling focus on, 314, 316–317
denormalizing/optimizing for, 316
identifying/modeling data based on, 313
in NoSQL data stores, 317–318
wide column storage example, 321, 324–325
account ID (user ID)
in binary search, 307
in full table scans, 305
implementing sharding key, 172–174, 188–189
partitioning search data, 325
search optimization, 309–310
ACID (Atomicity, Consistency, Isolation, Durability) transaction, 177, 178
active data set size, 167–168
ActiveMQ
advanced routing rules, 264
comparing messaging platforms, 286, 288, 292–294
load handling, 269
messaging protocols, 265–266
ordering messages, 280
overview of, 291–292
partial message ordering guarantee, 279
poison message handling, 284
ad hoc design sessions, 354
administration, Cassandra, 202–203
Advanced Message Queuing Protocol (AMQP), 265, 288–289
agent, monitoring, 342
agile teams, scaling
adding more people, 357–359
culture of alignment, 360–361
overview of, 357
procedures and innovation, 359–360
AJAX
front-end applications with, 29
local device storage for SPAs, 229
simplifying asynchronous processing, 252–253
alerts
automating, 340–345
automating build and deployment, 340
custom routing rules for, 264
alignment, culture of, 361–362
alter table statements, MySQL replication, 169
Amazon
CloudFront, 17, 117–119, 222–223
CloudSearch, 329
CloudWatch Logs, 346
Dynamo. See Dynamo data store
Elastic Cache, 238–239
RDS (Relational Database Service), 170
Route 53 service, 101–103, 117–119
Amazon EC2 (Elastic Compute Cloud)
auto-scaling, 115–116
AWS web deployment, 117–119
deploying own caching servers, 238–239
load balancer. See Amazon ELB (Elastic Load Balancer)
queue workers, 268
Amazon ELB (Elastic Load Balancer)
automating build and deployment, 339
auto-scaling, 116–118
as hosted service, 106–107
stateless web service machines and, 140
Amazon Machine Image (AMI), 115, 339
Amazon SQS (Simple Queue Service)
comparing messaging platforms, 293–294
disadvantages of, 288
message ordering problem, 280, 288
overview of, 285–288
poison message handling, 284
AMI (Amazon Machine Image), 115, 339
AMQP (Advanced Message Queuing Protocol), 265, 288–289
anti-patterns, message queue, 282–284
Apache mod_proxy, 224
Apache server, 53–54
Apache Traffic Server, 224
API-first design, web services, 127–129, 130–131
APIs (application programming interfaces)
learning simplicity from, 42–43
promoting simplicity with TDD, 41
in web application development, 124
web service design not using, 124–127
Apollo, 292
application architecture
front end, 28–30
overview of, 27–28
supporting technologies, 34–35
web services, 30–34
application metrics, reducing MTTR, 342
application servers
caching objects directly on, 231, 232
deploying web services in parallel to, 25
distributed storage/delivery of public files, 94
object cache servers using front-end, 25
placing reverse proxy servers, 220
serving private files, 95
stateful web, 86
using locks, 98–99
in web application layer, 24
applications
caching objects. See object caches
promoting loose coupling, 44–46
architecture
application. See application architecture
Cassandra, 199–204
event-driven. See EDA (event-driven architecture)
flexibility of good, 316
function-centric web services, 131–134
resource-centric web services, 134–138
Astyanax Chunked Object Store, Netflix, 96
asynchronous nature, of MySQL replication, 158–159
asynchronous processing
comparing messaging platforms, 284–294
currently in renaissance phase, 301–302
direct worker queue interactions as, 296–297
event-driven architecture as, 295–301
example of, 249–254
message queue anti-patterns, 282–284
message queue benefits, 270–276
message queue challenges, 276–282
message queues in, 256–270
overview of, 246
shopping analogy of, 254–256
summary, 301–302
synchronous example, 247–250
synchronous processing vs., 246
Atlassian Bamboo, 337
at-least-once delivery, message requeueing, 280
atomic counters, application-level sharding, 184
atomic transactions, ACID, 177
authorization, stateless web services for, 141–142
automation
build and deployment, 49, 335–340
Cassandra, 202–203
log aggregation, 345–347
monitoring and alerting, 340–345
overview of, 332–333
scaling with load balancers, 105
self-healing repair process, 79–80
testing, 333–335
auto-scaling
front-end scalability with, 114–116
with load balancers, 105–107
stateless web service machines with, 140
availability
CAP theorem and, 191–192
message queues evening out traffic spikes for, 273–274
MySQL ring replication reducing, 164–165
reduced thoroughput vs., 274–275
Azure
auto-scaling with, 115
Blob Storage for files, 93–95
Diagnostics, 346
load balancer as hosted service in, 106–107
scalability limits in Queues, 269
Search, 329
Service Bus Queues, 269
SQL Database Elastic Scale, 184
B
back-end server termination, ELB, 107
backpressure feature, RabbitMQ, 293
backup, MySQL replication, 159–160
bandwidth, CDN reducing, 15
benchmarks, gaining value of, 113
best practices, continuous deployment pipeline, 339–340
big data, 25
Big O notation, 305, 307–308, 328
Big-IP, F5, 109–111
binary search algorithm, 306–307
bindings, RabbitMQ, 264
binlog file, MySQL replication, 157–158, 161
Blob Storage, Azure, 93–95
blocking, 248–249
blocking I/O, 248
book index structure, 306, 328
book references, for this book, 364–366
browser cache
scalability of, 223
scaling front end with, 113–114
buffers, active data set size, 168
build process, automating, 49, 335–340
burnout, preventing, 348–349
business continuity plans, 79
business logic
front-end servers freed of, 22–24, 29
hexagonal architecture and, 32
pushing to web services, 24–25, 30–34
web application layer freed of, 24
business metrics, reducing MTTR, 342
C
cache hit ratio
bundling CSS/JS files to maximize, 216
overview of, 208–209
scaling reverse proxies, 224, 227
cache invalidation, 233, 243–244
cache-aside caches. See object caches
Cache-Control HTTP header, 213–217
cached objects. See HTTP-based caches
caches co-located with code, 230–232
caching
application objects. See object caches
AWS web deployment example, 117–119
cache hit ratio. See cache hit ratio
data partitioning using, 76–77
definition of, 12
front-end components using back end, 29
HTTP-based. See HTTP-based caches
local server, 97–98
Nginx benefits, 108
overview of, 208
preventing MySQL replication timing issues, 169
rules of thumb for, 239–244
scaling front end using, 113–114
summary, 244
caching proxy, 219–220
caching servers
caches co-located with code, 230–232
distributed, 232–233
scaling object caches, 238
call stack
asynchronous processing as programming without, 281
caching high up, 239–240
latency dictated by weakest link in, 296
callback
asynchronous call with, 251–254
definition of, 249
shopping analogy with, 255–256
Camel, integration with ActiveMQ, 264, 291–292
CAP (consistency, availability, and partition tolerance) theorem, 191–192
capacity, increasing
front line components, 22–24
load balancer limits, 109
load balancers increasing, 105
scalabilty and, 3
scaling horizontally, 16–17
scaling vertically, 8–10
cardinality, estimating for index, 308–310
Cassandra
failure handling, 80
scaling own file storage/delivery, 96
self-healing strategies, 196
topology, 199–204
as wide columnar data store, 317, 319–325
CDNs (content delivery networks)
caching front end, 113
definition of, 14
delivering public files to end users, 94
as front-end layer component, 101
horizontal scaling with, 17
hosting own hardware with, 121
as HTTP-based cache, 221–222
offloading traffic to third party with, 13
reducing bandwidth with, 15
scalability of, 223
scaling for global audience, 20–21
central processing unit. See CPU (central processing unit)
centralized log service, streaming logs to, 345
Chaos Monkey, 78
Chrome, 53–54
circular dependencies, 47
class diagrams, 59–60
classes
avoiding unnecessary coupling, 47
dependencies of, 60
dependency injection principle, 65–68
designing, 53
promoting loose coupling, 44–46
single responsibility principle of, 61–63
clients
decoupling providers and, 51–54
interacting with HTTP read-through caches, 211–212
in request/response interactions, 298–299
stateless web service machines and, 139
client-side caches
caching high up call stack, 240
overview of, 228–229
scaling, 233–234
client-side conflict resolution, 195–196
clones
implementing distributed locking, 99–100
load balancing by adding, 104–105, 108, 110
multiple slave machines and, 158
in publish/subscribe method, 263
replication challenges, 166
reverse proxy as independent, 226
scaling REST web services by adding, 138, 140
scaling web cluster by adding, 232
using local/independent locks, 98–99
Cloud Load Balancers, Rackspace, 111
cloud-based hosting
auto-scaling with load balancers, 105
AWS web deployment, 117–119
Azure SQL Database Elastic Scale, 184
file storage, 93–95
in, 268
load balancer, 106–107
log aggregation, 346
MySQL, 170
stateless web service machines, 140
CloudFront, Amazon
AWS web deployment, 117–119
cost effectiveness of, 17
delivering static/dynamic content, 222–223
CloudSearch, Amazon, 329
CloudWatch Logs, 346
clusters
reverse proxy in front of web service, 220–221
scaling by adding clones to web, 232
scaling distributed object caches, 233, 235–239
coarse locks, 143
code
80/20 rule for, 352–353
problems from hacking own, 50
reducing coupling when writing, 45
reviews, 355
writing vs. reusing, 49
collocated servers, 13
column families, Cassandra, 321–322
Common Object Request Broker Architecture (CORBA), 132
communication paths, 357–358
complexity
application-level sharding and, 184
dependency injection reducing, 68
key-value stores offering least, 317
message queue challenges, 281
promoting simplicity by hiding, 38–40
reducing coupling by hiding, 44
reducing with single responsibility, 61–63
shared lock management service increasing, 99
of SOAP, 134
composition, and repetitive code, 50
compound (composite) indexes
Cassandra tables similar to, 319–320
definition of, 311
ordering of columns in, 311–312
structuring data as, 325
wide columnar data stores using, 317
concurrency, measuring for higher, 3–4
conflict resolution
client-side, 195–196
of data store with CAP theorem, 191–192
eventual consistency and, 194–195
self-healing strategies for, 196
connection: close response header, 212
connection draining, Elastic Load Balancer, 107
consistency
ACID transactions, 177
CAP theorem and, 191–192
local application cache issues, 232
quorum, 196–197
rise of eventual, 192–197
trading high availability for, 197–199
consistent hashing, 236–237
constraints, scalability, 4
content delivery networks. See CDNs (content delivery networks)
continuous delivery, 336
continuous deployment pipeline, 336–340
continuous integration, 336
contract, decoding to, 51–54
contract surface area, 298, 299
cookies
establishing HTTP session, 88–89
handling session state with load balancers, 92–93
session data stored in, 89–90
copy and paste programming, 49, 50–51
CORBA (Common Object Request Broker Architecture), 132
core concepts
application architecture. See application architecture
data center infrastructure, 22–26
defining scalability, 2–4
evolution stages. See evolution to global audience
organizational scalability, 4
scalability in startup environment, 2
cost
benefits of ELB, 106
challenges of sharding, 176–184
of cookie-based session storage, 89–90
of hardware load balancers, 110
of hosting on own servers, 119
influencing, 354–355
manual vs. automated testing, 333–334
monolithic web service design and, 126–127
per user/transaction decreasing over time, 332–333
as project management lever, 349–350
saving with Amazon SQS, 285–286
of scaling by adding clones, 74
vertical scalability issues, 9–10
vertical vs. horizontal scaling, 16–17
Couchbase, 318
country code, sharding by, 174–175
coupling
avoiding producer/consumer, 283
class diagrams visualizing, 59
definition of, 43
dependency injection reducing, 65–68
direct worker queue interactions and, 296–297
in event-driven architecture, 299
loose. See loose coupling
measuring with contract surface area, 298
in request/response interaction, 296, 297–299
single responsibility reducing, 61
CPU (central processing unit)
function-centric web services and, 132
memory caches, 208
vertical scalability issues, 9–10
virtual private server upgrades, 7
Crash-Only, 78–79
critical path, message queues, 271
cron-like consumer approach, 261
culture of alignment, engineers, 361–362
custom routing rules, consumers, 264–265
customers, single server set-up, 6
D
daemon-like consumer approach, 261
data
consistency in MySQL replication, 169
redundancy, from denormalization, 316
scalability issues of more, 3
searching for. See searching for data
data center infrastructure
additional components, 25
data persistence layer, 25–26
front line, 22–24
overview of, 22
understanding, 26
web application layer, 24
web services layer, 24–25
data centers
in content delivery network, 14–15
deployment of private, 119–121
in horizontal scalability, 18
in isolation of services, 14
load balancers as entry point to, 103
Route 53 latency-based routing and, 101–102
scaling for global audience with multiple, 19–21
data layer
MySQL. See MySQL, scaling
NoSQL. See NoSQL data stores, scaling
overview of, 156
partitioning. See data partitioning (sharding)
summary, 204
data model, Cassandra, 201
data normalization, 189–190
data partitioning (sharding)
advantages of, 175–176
building own file storage/delivery, 96
Cassandra automatic, 200–201
challenges of, 176–184
choosing sharding key, 171–175
implementing, 188–189
overview of, 170–171
putting it all together, 184–189
scaling cache cluster using, 239
scaling distributed object caches, 235–237
storing session data in dedicated session store, 91
wide columnar data stores using, 317
data persistence layer, data center, 25–26
data set size
affecting cache hit ratio, 208–209
master-master replication and, 164
reducing in indexes for faster search, 306
replication challenges, 166–170
splitting in data partitioning. See data partitioning (sharding)
data storage, 113–114
data stores
advantages of sharding, 176
in application architecture, 34–35
horizontal scalability/high availability for, 190
logging to, 345
mapping data with, 180–182
NoSQL era and, 191
replication of. See replication, MySQL
rise of eventual consistency for, 192–197
scaling object caches vs. scaling, 239
scaling own file storage/delivery, 96
storing session data in dedicated, 90–91
databases
avoid treating message queues as, 282–283
front end unaware of, 29
scaling by adding clones, 73
Datadog monitoring service, 344
DCOM (Distributed Component Object Model), 132
decision-making, 351
decoupling
API-first design for web services, 128
clients/providers, 51–54
definition of, 44
in event-driven interactions, 297
message consumers, 259
message queues promoting, 275–276
in MySQL replication, 158
publisher/consumer in RabbitMQ, 288–289
dedicated search engine, using, 328–330
DELETE method, HTTP, 135–136, 211
deletes
avoiding message queue, 283
distributed object cache and, 232
as limitation in Cassandra, 203–204
local application cache issues, 232
delivery, continuous, 336
denormalization, 316, 317, 325
dependencies
class diagrams visualizing, 59–60
promoting loose coupling and, 44
reducing between teams, 359
dependency injection, 65–71
deployment
automating process of, 49, 335–340
front-end layer examples, 117–121
design. See software design principles
design patterns
in copy-paste programming, 50–51
drawing diagrams with, 60
for publish/subscribe model, 264
using Amazon SQS with, 286
Diagnostics, Azure, 346
diagrams
circular dependencies exposed via, 47
class, 59–60
drawing, 54–57
module, 60–61
reducing coupling on higher levels of abstraction via, 45
use case, 57–58
direct worker queue interaction, and EDA, 296–297
direct worker queue model, routing, 262
disaster recovery plans, 79
Distributed Component Object Model (DCOM), 132
distributed locking
implementing, 98–101
Terracotta allowing for, 91
web service challenges, 142–143
distributed object caches
cache high up call stack, 240
overview of, 232–234
scaling, 235–236
distributed transactions, 178
DNS (Domain Name System) server
in CDNs, 15
in client caches, 208
as front-end component, 102–103
in front line of data center infrastructure, 22–23
geoDNS server, 19–23
in isolation of services, 12, 14
in round-robin DNS service, 18–19
in round-robin–based load balancing, 103–104
in single server set-up, 5–6
in vertical scalability, 11
document IDs, inverted index structure, 327–328
documentation
80/20 rule for, 352–353
reducing complexity with message route, 281
via diagrams. See diagrams
document-oriented data stores, NoSQL, 317–318
downloading files, 93–96
drafts, diagram, 55–57
draw diagrams, as software design principle, 54–61
draw.io, drawing diagrams with, 60
DRY (don’t repeat yourself), software design principle, 48–51
duplication
API-first design for web services with, 127–129
avoiding in software design, 48–51
local application cache issues, 232
durability, ACID transactions, 177
dynamic content, CDN, 222–223
dynamic languages, front end, 111
Dynamo data store
as key-value data store, 317
pushing conflict resolution onto clients, 195
scaling with NoSQL, 190–193
E
eBay bidding application
local locks preventing scaling out in, 98–99
scaling by adding clones, 72–74
scaling with data partitioning, 75–77
scaling with functional partitioning, 74–75
EC2. See Amazon EC2 (Elastic Compute Cloud)
EDA (event-driven architecture)
currently in renaissance phase, 301–302
direct worker queue interaction, 296–297
event-based interaction, 297–301
request/response interaction, 296
traditional interactions vs., 32–33
edge-cache servers, 20–21
Elastic Cache, Amazon, 238–239
Elastic Compute Cloud. See Amazon EC2 (Elastic Compute Cloud)
ELB. See Amazon ELB (Elastic Load Balancer)
asynchronous processing of, 250–254
class diagram, 59–60
single responsibility for validation of, 62–63
synchronous processing of, 247–250
EmailService interface, 60
engineering department, scaling, 332, 349, 357–361
equal distribution, index performance, 310–311
ESB (enterprise service bus), 260
event sourcing, EDA, 300–301
event-based interaction, EDA, 297–301
event-driven architecture. See EDA (event-driven architecture)
events, definition of, 295
eventual consistency, 193–197, 203
evolution to global audience
content delivery network, 13–16
horizontal scalability, 16–19
isolation of services, 11–13
overview of, 5
scalability for global audience, 19–21
single-server configuration, 5–7
vertical scalability, 7–11
exactly-once delivery, message requeueing, 280
exchange concept, RabbitMQ, 289
Extensible Markup Language-Remote Procedure Call (XML-RPC), 132
F
failover
Azure automatic, 239
MySQL maintenance timeline for, 162–163
MySQL master-master, 161–162
MySQL not supporting automatic, 160
NoSQL with automatic, 198
removing for high availability, 79
using load balancers with automatic, 106
failure handling
with load balancers, 105
for messaging platform, 284
with MySQL replication, 159–162
with stateless web service machines, 139
fearless engineers, 334
feature toggles, build and deployment, 339–340
features
80/20 rule for new, 352
Amazon SQS, 288
RabbitMQ, 291
feedback
balancing schedule with, 356–357
continuous deployment pipeline, 339
Lean Startup methodology, 285
making use of, 49
ongoing code reviews, 355
releasing smaller chunks for customer, 356
ActiveMQ for messaging, 292
message queue as, 283
solving message ordering with, 278
file storage
choosing deployment, 121
managing, 93–96
as possible single point of failure, 79
using Amazon S3 for, 119
file system, scaling local cache, 235
file-based caches, 235
fine-grained locks, 143
fire-and-forget requests
asynchronous example, 249–251
easier scalability and, 272
Firefox, 53–54
First In First Out. See FIFO (First In First Out)
flexibility
Amazon SQS disadvantages, 288
of good architecture, 316
of RabbitMQ, 288–291
framework, front end, 111
front cache servers, 22–24
front-end layer
application architecture for, 28–30
building, 84–85
deployment examples, 117–121
overview of, 84
summary, 121
front-end layer, scalability components
auto-scaling, 114–116
caching, 113–114
DNS, 102–103
load balancers, 103–111
overview of, 101–102
web servers, 111–113
front-end layer, state
for files, 93–96
for HTTP sessions, 88–93
other types of, 97–101
stateless vs. stateful services, 85–88
frontline layer, data center infrastructure, 22–24
full page caching, 220–221
full table scans, 305
full text search, with inverted indexes, 326–328
functional partitioning
with distributed locking, 98–100
isolation of services using, 13–14
scaling with, 71, 74–75, 185–187
function-centric web services, 131–135
functions
MySQL replication, 169
sharding, 182
G
generic server metrics, reducing MTTR, 341
GET method
caching service responses, 146–148
challenges of sharding, 176–177
HTTP and resource-centric services, 135–138
HTTP session management, 88, 90–91
GFS (Google File System), 96, 190–191
github web hook, 338
global audience, scalability for, 19–21
globally unique IDs, application-level sharding, 184
Google Maps API, 42–43
Google Trends, 292–293
Grails in Action, 42
GridFS, in MongoDB, 96
group, auto-scaling, 115–116
H
HA Proxy, 119–120
Hadoop, 42
Hadoop Distributed File System (HDFS), 96
Hadoop in Action, 42
HAProxy, 107–109
hard drive caches, 208
hard drive speed, 9–10
hardware
isolation of services using rented, 13
load balancers, 109–111
private data center hosting own, 119–121
reverse proxy, 226–227
upgrading for vertical scalability, 8–9
HBase, 317
HDFS (Hadoop Distributed File System), 96
headers, HTTP, 211–217
hexagonal architecture, 32
high availability
building own file storage/delivery, 95
comparing in messaging platforms, 293
data stores for, 190–191
definition of, 78
Elastic Load Balancer with, 106
eventual consistency increasing, 194
HAProxy with, 109
MySQL replication with, 159–160
software design for, 77–80
trading for consistency, 197–199
high cardinality fields, indexes, 309–310
high value processing, message queues, 271
Hollywood principle, IOC as, 69
horizontal scalability
Cassandra for, 203
comparing in messaging platforms, 294
data partitioning and. See data partitioning (sharding)
data stores for, 190–191
deferring building of, 353
evolution to, 16–19
RabbitMQ for, 291
scaling data layer for. See data layer
stateless front-end web servers for, 111–112
wide columnar data stores using, 317
HTML (Hypertext Markup Language), 124, 217
HTTP (Hypertext Transfer Protocol)
coding to contract and, 53–54
edge-cache servers and, 20
managing sessions, 88–93
REST services using, 135–137
in single server set-up, 5–6
testing with Jmeter, 335
web applications initially built with, 124
HTTP-based caches
caching headers, 211–217
caching proxies, 219–220
caching rules of thumb, 239–244
CDNs, 221–222
between clients and web service, 213
object caches vs., 227
overview of, 210–211
reverse proxies, 220–221
scaling, 223–227
SOAP scalability and, 134
types of, 217–218
HTTPS (HTTP over TLS Transport Layer Security), REST services, 138
hybrid applications, front-end layer, 85
Hypertext Markup Language (HTML), 124, 217
I
idempotent consumer, 280
IIS (Internet Information Services), 53–54
incremental change, for inefficient processes, 49
indexes
adding overhead, 308–309
binary search algorithm in, 306–307
book, 305–306
compound (composite), 311
estimating field cardinality for, 308–310
full table scans vs., 305
full text search using inverted, 326–327
item distribution in, 310–311
key-value stores not supporting, 317
as lookup data structure, 305
properties of, 305–306
searching for data and, 304–305
using job queue for search engine data, 329
infrastructure, messaging, 266–270
inheritance, for repetitive code, 50
innovation, scaling engineering department, 359–361
integration, continuous, 336
integration tests, 335
interaction rates, scaling for higher, 3–4
interfaces
dependencies of, 60
in open-closed principle, 63–64
Internet Information Services (IIS), 53–54
interoperability, JMS/STOMP, 266
inverted indexes, 326–330
I/O (input/output)
blocking, 248
as indexing overhead, 308
in MySQL replication, 158–159
nonblocking, 253
vertical scalability improving, 8–9
IOC (inversion of control), software design principle, 68–71
IP (Internet Protocol) address, 5–6, 101–102
isolation
in ACID transactions, 177
decoupling of producers/consumers, 275–276
evolution of services to, 11–13
message queues for failure, 274–275
of queue workers, 268
J
Java
ActiveMQ written in, 291–292
distributed locking in, 99–100
overengineering in, 40
using inversion of control, 68
Java JVM, 91
JavaScript, 228–229
Jenkins, 337–338
JMS (Java Message Service), 266, 292
JMX (Java Management Extensions) protocol, ActiveMQ, 292
job queue, search engines, 329
JSON (JavaScript Object Notation)-based REST services, 133, 136
Kafka topic partitioning, 279
keep-alive header, 212
keys
accessing value in web storage, 229
client-side caches and, 228
key-value stores
client-side caches as, 228–229
distributed object caches as, 232
NoSQL, 317
Kibana web interface, Logstash, 347
L
language
function-centric web services and, 132
selecting for front end, 111
latency
Amazon Route 53 routing and, 102
dictated by weakest link in call stack, 296
eventually consistent stores and, 197
hosting own hardware and, 119
shared lock management increasing, 99
LbaaS load balancer, Open Stack, 111
Lean Startup methodology, 285, 356
Least Recently Used (LRU) algorithm, 224, 233
links, references, 374–377
Linux OS file caches, 208
Load Balancer, Azure, 111, 140
load balancers
benefits of, 104–106
benefits of stateless web service machines, 139–140
deploying private data center with, 119–120
DNS round-robin–based, 103–104
as front line of data center infrastructure, 22–24
as front-end layer component, 101
handling session state with, 92–93
hardware-based, 109–111
as hosted service, 106–107
in MySQL replication with multiple slaves, 158
self-managed software-based, 107–109
load testing, Jmeter, 335
local cache
caching in different layers of stack, 240
implementing, 230–232
scaling web server, 235
local device storage, client-side cache, 228–229
local simplicity, in software design, 39–40
lock contention, 9–10
locks
managing server state, 98–99
preventing deadlocks, 142
resource. See resource locks
logging
automating log aggregation, 345–347
custom routing rules for, 264
log-forwarding agents, 346
Loggy, hosted log-processing service, 346
Logstash, 346–347
longevity, affecting cache hit ratio, 209
loose coupling
avoiding unnecessary coupling, 47
models of, 47–48
overview of, 43–44
promoting, 44–46
low value processing, message queue, 271
LRU (Least Recently Used) algorithm, 224, 233
M
maintenance
cloud service provider costs for, 17
data stores for file storage reducing cost of, 96
higher costs for more code, 50
load balancers for hidden server, 104
master-master deployment for long-lasting, 161–163
message queues and performing, 274
stateless web services for easy, 139
manual deployment, vs. automated, 335–336
manual testing, vs. automated, 333–334
mapping
keeping data in separate database, 179–182
modulo-based issues, 178
multidatabase sharded solution, 182
scaling with data partitioning using, 76–77
sharding key to server number, 172–173
master server
MySQL master-master replication, 161–164
MySQL replication, 169–170
MySQL ring replication, 164–165
replicating sharding key mappings, 180–182
master-master replication, MySQL
adding difficulty by using, 166
challenges of, 166–170
deploying, 160–163
not viable for scalability, 163–164
master-slave topology, MySQL
object caches allowing for replication, 237
recovering from failure, 160–161
replication, 157–159
replication, scaling cache cluster, 237–238
replication challenges, 166–170
as single source of truth semantics, 166
max-age response, Cache-Control HTTP header, 214
Memcached
distributed locking, 100–101
distributed object caches, 232–234
scaling distributed object caches, 235–236
memory
cache servers using LRU for limits of, 233
implementing local application caches, 230
as indexing overhead, 308
needs of search engines, 328
message brokers
in ActiveMQ, 291–292
creating custom routing rules, 264–265
isolating failures, 274–275
in message queue-based processing, 259–260, 273–274
metrics, 267–270
in RabbitMQ, 290
scaling horizontally, 268
in system infrastructure, 267
message consumers
benefits of message queues, 272–273
custom routing rules for, 264–265
decoupling producers from, 260, 274–275, 283
delivering messages to, 256–257
direct worker queue interaction, 262, 297
event-driven interaction, 297, 299
idempotent, 280–281
message ordering problem, 276–279
messaging infrastructure for, 268–269
overview of, 260–262
publish/subscribe method, 263–264
message groups, ActiveMQ, 279, 292
message of death, handling, 284
message ordering problem
causing race conditions, 281
overview of, 276–278
partial message ordering, 279
solving, 278–279
message producers
decoupling consumers from, 260, 275–276, 283
in direct worker queue interactions, 297
in event-driven interactions, 297, 299
isolating failures and, 274–275
overview of, 258–259
message publishing, 258, 274–276
message queues
anti-patterns, 282–284
benefits of, 270–276
caching high up call stack, 240
challenges of, 276–282
as data center infrastructure, 25
example of, 250–254
front-end sending events to, 29
message broker, 259–260
message consumers, 260–265
message producers, 258–259
messaging infrastructure, 266–270
messaging protocols, 265–266
overview of, 256–257
removing resource locking in web services, 142
message requeueing problem, 280
message-oriented middleware (MOM), 259–260
messaging infrastructure, 266–270
messaging platforms
ActiveMQ, 291–292
Amazon SQS, 285–288
final comparison notes on, 292–294
overview of, 284–285
RabbitMQ, 288–291
messaging protocols, 265–266, 288
metatags, avoiding cache-related HTML, 217
metrics, reducing MTTR, 341–343
Microsoft Azure. See Azure
minimal viable product (MVP) development, Lean Startup, 285
mobile clients
developing mobile APIs, 124
scaling front end with browser cache, 113–114
single-page application for devices, 229
mocks, startup development, 357
modeling data
NoSQL, 313–318
overview of, 313
wide column storage example, 318–325
modules
avoiding unnecessary coupling in, 47
class diagrams of, 59–60
drawing diagrams, 60–61
loose coupling for, 44–46
single responsibility for, 62
modulo-based mapping, and sharding, 178
MOM (message-oriented middleware), 259–260
MongoDB
as document-oriented data store, 318
fast recovery for availability, 197–199
scaling file storage/delivery with, 96
monitoring
automating, 340–345
installing agent on each server, 342
tools, 340
monolithic application with web service, 124–127, 130–131
MTTR (mean time to recovery), reducing
in monitoring and alerting, 340–345
in self-healing, 80
multidatabase sharded solution, 181–182, 183
multilayer architecture, 31
multiple load balancers, 109–110
multiple reverse proxy servers, 225–226
must-revalidate response, Cache-Control HTTP header, 214
MVP (minimal viable product) development, Lean Startup, 285
MySQL, as most popular database, 156
MySQL, scaling
overview of, 156
replication, 156–166
replication challenges, 166–170
vertical scalability issues, 9–10
N
NASA (National Aeronautics and Space Administration), 50
Netscaler, Citrix, 109–111
networks
HTTP proxy server in local, 219–220
improving throughput for vertical scalability, 9
Nginx
private data center deployment using, 119–120
reverse proxy, 224
as software-based load-balancer, 107–109
superior performance of, 226
no-cache response, Cache-Control HTTP header, 214
nodes
in Cassandra topology, 80, 199–201
in MongoDB failure handling, 198–199
share-nothing principle for, 76
nonblocking I/O, 253
noncacheable content, HTTP headers of, 216–217
normalization
NoSQL denormalization, 316
in relational data model, 314–315
NoSQL data stores
data as index in, 312–313
in data layer of data center, 25
data modeling, 313–317
dedicated search engine for, 328–330
defined, 190–191
as most commonly used, 317–318
NoSQL data stores, scaling
Cassandra topology, 199–204
faster recovery for availability, 197–199
overview of, 189–191
rise of eventual consistency, 192–197
no-store response, Cache-Control HTTP header, 214
no-transform response, Cache-Control HTTP header, 214
O
OASIS (Organization for the Advancement of Structured Information Standards), AMQP, 265
object caches
caching application objects, 227–228
caching rules of thumb, 239–244
client-side, 228–230
co-located with code, 230–232
distributed object, 232–234
scaling, 234–239
size affecting cache hit ratio, 209
object-clustering, Java JVM session storage, 91
object-oriented languages, coupling in, 44–45, 47
open-closed principle, 63–68
operating system
metrics reducing MTTR, 341
as multilayered architecture, 31
operations, scalability of, 332
optimistic concurrency control, 142
OR conditions, full text search, 328
Organization for the Advancement of Structured Information Standards (OASIS), AMQP, 265
organizational scalability, constraints, 4
overengineering
API-first for web services risking, 128–129
avoiding for simplicity, 40–41
designing for scale without, 71
overhead, added by indexes, 308–309
overtime, and productivity, 347–348
P
pair programming, 354
parallel back-end processing, 272–273
partial message ordering guarantee, 279
partition tolerance, CAP theorem, 191–192
partitioning. See data partitioning (sharding)
partitioning, topic, 279
pattern matching, customizing routing rules for, 264
performance
asynchronous processing and, 253–254
caching to improve, 242–243
increasing. See caching
synchronous processing and, 249–250
persistent data, and stateless web services, 140–141
pipes, Unix command-line program, 47–48
plugins, inversion of control principles for, 70
poison message handling, 284
policies, scaling, 115
POST method, HTTP, 135–136, 211
pragmatic approach, web service design, 130–131
presentation layer, and web services, 124–127, 130–131
primary node failure, MongoDB, 198–199
prioritizing
tasks to manage scope, 351–354
where to start caching, 242–243
private response, Cache-Control HTTP header, 213
procedures, scaling engineering department, 359–361
processes, wasting time on inefficient, 49
productivity, scaling. See automation; yourself, scaling
products, building teams around, 358
protocols, messaging, 265–266
providers
auto-scaling using hosting, 115
coding to contract to decouple clients from, 51–54
configuring/scalability of CDN, 221–223
proxy (intermediate), HTTP-based caches, 210
proxy servers, 223
pt-table-checksum, MySQL replication issues, 170
pt-table-sync, MySQL replication issues, 170
public files, 93–94
public response, Cache-Control HTTP header, 214
publishing, message, 258, 274–276
publish/subscribe queue model, routing, 263–264
PUT method, HTTP, 135–136, 211
Q
queries
in Cassandra, 202
designing NoSQL data model, 314–316
executing across shards, 176–177
optimizing for kinds of, 325
wide column storage example of, 318–321
queue workers
allowing self-healing of system, 274–275
isolating, 268
scalability by adding parallel, 272–273
in system infrastructure, 267–268
queue-based interaction, and EDA, 296–297
queues, in Cassandra, 204
quorum consistency, 196–197, 203
R
RabbitMQ
comparing messaging platforms, 286
flexible routing rules in, 264
message ordering problem, 280
messaging infrastructure, 269
messaging protocols for, 265–266
overview of, 288–291
poison message handling in, 284
Rackspace
auto-scaling with, 115
hosting MySQL with Cloud Database, 170
RAID (Redundant Array of Independent Disks), 8, 95–96
Rails, 68
RAM (random access memory), 7–10
random access, in message queue, 283
random access I/O, 8
random access memory (RAM), 7–10
random order, solving message ordering, 278
rapid learning, Lean Startup, 356
RDS (Relational Database Service), Amazon, 170
read-only statements, MySQL replication, 158
reads
adding replica servers, 185–186
eventual consistency conflicts, 194–196
MySQL replication, 166, 186–188
MySQL replication timing issues, 169
trading high availability for consistency, 197–199
read-through caches
cache-aside caches vs., 227
caching proxies as, 219–220
HTTP-based caches as, 210–212
Redis, distributed object caches, 232–234
redundancy, for high availability, 79–80
Redundant Array of Independent Disks (RAID), 8, 95–96
refactoring
80/20 rule for, 353
copy-paste programming and, 50
single responsibility principle for, 61–63
references, for this book
books, 364–366
links, 374–377
talks, 373–374
white papers, 366–373
regression testing, 333
reinventing the wheel, avoiding wasted time, 49
relational data model, 313–315
Relational Database Service (RDS), Amazon, 170
relay log, MySQL replication, 158, 161
release cycle
reducing size of each, 356–357
wasting time on inefficient processes in, 49
remote servers, message queue interaction with, 271
replica servers, adding, 185–186
replica sets, MongoDB, 198–199
replication
Cassandra, 201–202
local caches not using, 231–232
scaling object caches with, 237–238
applicable to other data stores, 170
challenges of, 166–170
defined, 156–157
handling slave failures, 160
master-master, 160–164
master-slave, 157–158
overview of, 156
ring, 164–165
scaling with, 186–187
summary of, 166
using multiple slaves, 158–159
request headers, HTTP-based caches, 212–213
request/response interaction, and EDA, 296
requirements, scalability, 3–4
resource intensive work, with message queues, 271
resource locality, CDNs, 15
resource locks, 98–99, 141–143
resource management, 105–106
resource-centric web services, 134–138
response headers, HTTP-based caches, 212–214
REST (Representational State Transfer) web services
between front-end and web services, 25
JSON-based, 133
as resource-centric, 135–138
REST API, RabbitMQ, 289–290
REST web services, scaling
caching service responses, 146–149
cluster of, 220–221
functional partitioning, 150–153
keeping service machines stateless, 139–146
overview of, 138
return channels, avoiding message queue, 282–283
reuse of cached objects, 240–242
reuse of code
avoid reinventing the wheel, 49
open-closed principle for, 64–65
single responsibility principle for, 61–63
reuse of tools, 355
revalidation, Cache-Control HTTP header, 214
reverse proxies
caching high up call stack, 240
as front-end layer component, 101
as HTTP-based cache, 220–221
managing scalability of, 223–227
scaling front end with caching, 113
as software-based load-balancers, 107–109
Riak, 317
ring replication, MySQL, 164–170
round-robin DNS service, 18, 19, 103–104
Route 53 service, Amazon, 101–103, 117–119
routing
Amazon SQS and, 288
methods, 262–265
RabbitMQ advanced message, 288–290
rows, Cassandra table, 319
rules
creating indexes, 310
custom routing, 264–265
rules, caching
cache invalidation is difficult, 243–244
caching high up call stack, 239–240
reusing cache among users, 240–242
where to start caching, 242–243
run-time environments, function-centric web services, 132
S
S3 (Simple Storage Service), 93–95, 117–119
scalability
ActiveMQ, 291–292
agile teams, 357–361
Amazon SQS, 286–288
automation. See automation
concept of, 2–4
definition of, 3
engineering department, 349
local application vs. distributed object caches, 232–234
message queues for easier, 272–273
of object caches, 234–237
operations, 332
RabbitMQ, 291
as software design principle, 71–77
startup environment and, 2
your own impact, 349
for yourself. See yourself, scaling
schedule, influencing, 355–357
schema, NoSQL data model, 314
scope
influencing by prioritizing tasks, 350–354
as project management lever, 349–350
Search, Azure, 329
search engines
introduction to, 326–328
memory needs of, 328
overview of, 326
using dedicated, 328–330
searching for data
introduction to indexing. See indexes
modeling data, 313–325
overview of, 304
search engines, 326–330
summary, 330
Secure Sockets Layer (SSL)
overview of, 220
termination. See SSL termination
REST services vs. SOAP, 138
stateless web services for, 140–142
Selenium, end-to-end tests, 335, 339
self-healing
auto-scaling similar to, 116
in Cassandra, 202–203
designing software for, 77–80
message queues promoting, 274–275
self-managed software-based load-balancers, 107–109
separation of concerns, API-first design for web services, 128
sequential I/O, 8
server number
mapping data in separate database, 179–182
mapping sharding key to, 179
servers
adding to sharded deployments, 178
automating configuration of, 338
horizontal scalability with multiple, 16–19
hosting on hardware vs. virtual, 119
isolating roles in functional partitioning, 74–75
isolating services to separate, 11–14
managing state, 97–101
reducing with CDNs, 15
stateless vs. stateful, 85–88
uploading user-generated content to, 93–96
Service Bus Queues, scalability limits of, 269
service calls, in Amazon SQS, 288
service level agreement (SLA), 343
service-oriented architecture. See SOA (service-oriented architecture)
services
adding abstraction for, 40
building teams around, 358
isolating to separate servers, 11–14
in request/response interactions, 299
scaling with functional partitioning, 74–75
in web services layer, 30–34
sessions, managing HTTP, 88–93
setters, unnecessary coupling and, 47
sharding. See data partitioning (sharding)
sharding key
choosing, 171–174
definition of, 171
implementing, 188–189
mapping to server number, 178
shared libraries, 50
shared memory, 230
shared object cache, 141–142
share-nothing principle
advantages of sharding, 175–176
scaling distributed object caches, 236
scaling with data partitioning using, 76–77
Simple Logging Facade for Java (SLF4J), loose coupling in, 48
Simple Mail Transfer Protocol (SMTP), 59–60
Simple Object Access Protocol. See SOAP (Simple Object Access Protocol)
Simple Queue Service. See Amazon SQS (Simple Queue Service)
Simple Storage Service (S3), 93–95, 117–119
simplicity
hiding complexity/building abstractions, 38–40
learning from software design, 42–43
overengineering reducing, 40–41
single responsibility increasing, 61–63
as software design principle, 38
with test-driven development, 41–42
single points of failure, 79, 106
single responsibility, 61–63, 68
single-page applications. See SPAs (single-page applications)
single-server configuration
adding vertical scalability to, 7–11
evolution from, 5–7
isolation of services for, 11–14
scalability limitations of, 7
scaling by adding copies of same thing, 184–185
size
of cache affecting cache hit ratio, 209
data set. See data set size
scalability of reverse proxy, 224
SLA (service level agreement), 343
slave servers, MySQL replication. See also master-slave topology, MySQL
breaking data consistency, 169–170
multiple, 158–159
overview of, 157–158
rebuilding of failed, 160
returning stale data, 169
scaling reads, 166
SLF4J (Simple Logging Facade for Java), loose coupling in, 48
SMTP (Simple Mail Transfer Protocol), 59–60
SOA (service-oriented architecture)
definition of, 30
RabbitMQ message routing as, 290
scaling with functional partitioning in, 74–75
SOAP (Simple Object Access Protocol)
as function-centric, 132
integration flow for, 132–133
interoperability/usability of, 133–134
over HTTP, 25
REST vs., 137–138
scalability issues of, 134
SOA vs., 30
software design principles
coding to contract, 51–54
dependency injection, 65–68
design for scale, 71–77
design for self-healing, 77–80
don’t repeat yourself, 48–51
draw diagrams, 54–61
inversion of control, 68–71
loose coupling, 43–48
open-closed principle, 63–65
overview of, 38
simplicity, 38–43
single responsibility, 61–63
summary, 81
software-based load-balancers, self-managed, 107–109
solid-state drives. See SSDs (solid-state drives)
Solr, as dedicated search engine, 329
sorting algorithm, open-closed principle, 63–64
SPAs (single-page applications)
building front-end layer as, 84
building front-end layer as hybrid application, 85
local device storage for, 229
scaling front end with browser cache, 113–114
Sphinx, as dedicated search engine, 329
spikes
ActiveMQ message, 292
message queues evening out traffic, 273–274
Spring framework, 68
Spring Recipes, 42
SpringSource, 292
SQL Database Elastic Scale, Azure, 184
SQS. See Amazon SQS (Simple Queue Service)
Squid, as open-source reverse proxy, 224
SSDs (solid-state drives)
building own file storage/delivery, 96
improving access I/O for vertical scalability, 8
scaling reverse proxies vertically, 227
SSL (Secure Sockets Layer), 220
SSL termination
benefits of Elastic Load Balancer, 106
defined, 106
HAProxy load balancer supporting, 109
stale, cached objects as, 214
startups, high failure rate of, 71
state, managing
files, 93–96
HTTP sessions, 88–93
keeping web service machines stateless, 139–146
other types of state, 97–101
stateless vs. stateful services and, 85–88
stateful services, stateless vs., 85–88
stateless services
defined, 73
queue workers as, 268
scaling by adding clones to, 73
stateful vs., 85–88
web servers as, 268
web service machines as, 139–146, 268
static files, 215–216, 222–223
STOMP (Streaming Text-Oriented Messaging Protocol), 265–266, 288
streaming logs, to centralized log service, 345
Streaming Text-Oriented Messaging Protocol (STOMP), 265–266, 288
subscription methods, message consumers, 262
subsets, in sharding, 171, 175
Symfony, and inversion of control, 68
synchronization
consistent data stores supporting, 196
local application caches not using, 231–232
replication in MySQL as, 157
synchronous invocation, as temporal coupling, 296
synchronous processing
affecting perceived performance, 249–250
asynchronous processing vs., 246
example of, 247–249
shopping analogy for, 254–255
T
tables, Cassandra, 201, 319–323
talks, references for, 373–374
tasks
80/20 rule for, 353
delegating responsibilities and, 354
prioritizing to manage scope, 351–354
TCP socket, avoid treating message queue as, 282–283
TCP/IP programming stack, 31
TDD (test-driven development), 40–41
technologies, application architecture supporting, 34–35
templates, 29
temporal coupling, 296–297
Teracotta, JVM session storage, 91
test-driven development (TDD), 40–41
testing, automated, 333–335
third-party services
application architecture, 35
content delivery network, 13–16
data centers, 24
Datadog monitoring service, 344
deploying private data center, 119–120
distributed file storage, 96
front end unaware of, 29
horizontal scaling, 17–19
hosting DNS, 101
integration between web applications, 25
reducing workload/increasing costs, 355
scaling HTTP caches, 223–227
service level agreement monitoring, 343
sharing application logs with, 346–347
time
avoiding overtime, 347–349
influencing schedule, 355–357
MySQL replication issues, 169
as project management lever, 349–350
time to discover, MTTR, 340
time to investigate, MTTR, 341
Time to Live. See TTL (Time to Live) expiration
time to respond, MTTR, 340–341
tokens, inverted index structure, 327–328
Tomcat, 53–54
traditional multipage web applications, 84, 85
traffic
benefits of stateless web service machines, 139
in CDNs, 15
distribution in single server, 5–6
message queues evening out spikes in, 273–274
scaling for global audience, 19–21
single-server scalability limits, 7
TTL (Time to Live) expiration
defined, 209
keeping web service machines stateless, 140
max-age response, Cache-Control HTTP header, 214
overcoming cache invalidation by setting short, 243–244
scalability of reverse proxy, 224
U
Ultima Online, 171
UML (Unified Modeling Language), 60
Unified Modeling Language (UML), 60
uniform resource locators. See URLs (uniform resource locators)
Unix command-line program, loose coupling, 47–48
updates
avoiding message queue, 283
breaking data consistency in MySQL replication, 169
denormalized data model issues, 316
stateless web service machines and, 139
validating data model use cases, 324–325
uploading, user-generated content to your server, 93–96
URLs (uniform resource locators)
bundling CSS and JS files under unique, 216
distributed file storage using S3, 94
downloading files using, 93
REST services using, 135–137
use cases
API-first design for web services, 127–129
drawing diagrams, 57–58
file storage, 93
message queues, 271
preventing repetition with most common, 51
stateless web services, 141–146
validating data model against known, 324–325
web server, 111–112
wide column storage example, 318–322
user ID. See account ID (user ID)
user interface, front end application as, 29
user-generated content, uploading to servers, 93
users
reuse of cached object, 240–242
sharding based on, 172–174
validating data model use cases, 324
V
value
accessing in web storage, 229
client-side caches and, 228
Vary HTTP header, 215
vertical scalability
cost issues of, 9–10
definition of, 8
methods, 8–9
reverse proxies, 227
system architecture unaffected by, 10–11
vision, validating for success, 351
VPS (virtual private server), 6, 7, 13
W
web application layer
data center infrastructure, 24
managing HTTP sessions, 91–92
master-master replication and, 163
sharding in, 174, 176–178, 188
web application servers, 240
web applications, building front-end layer, 84–85
web browsers, decoupling from providers, 53–54
web flows, separate from business logic, 29
web servers
benefits of load balancers, 104–106
decoupling from clients, 53–54
for front-end scalability, 111–113
as front-end layer component, 101
for HTTP session with cookies, 88–89
local cache on, 230–232
reverse proxy reducing load on, 220–221
scaling by adding clones, 72–74
scaling local caches, 235
web services
application architecture for, 30–34
in data center infrastructure, 24–25
designing, overview, 124
designing API-first, 127–129
designing as alternative presentation layer, 124–127
designing with pragmatic approach, 130–131
function-centric, 131–134
overview of, 124
resource-centric, 134–138
REST. See REST (Representational State Transfer) web services
scaling REST. See REST web services, scaling
scaling with functional partitioning, 74–75
summary, 153
web session scope, HTTP sessions, 89
web storage
with JavaScript code, 228–229
scaling client-side caches for, 233–234
speeding up application time, 229–230
using as cache vs. reliable data store, 229
white paper references, 366–373
wide columnar data stores, 317, 318–325
writes
Cassandra optimization for, 202, 322–323
cost of deletes in Cassandra, 203–204
eventual consistency conflicts, 194–196
master-master replication and, 163–164
not scaling using replication, 166
scaling with, 186–187
trading high availability for consistency, 197–199
ws-* specifications, SOAP, 133–134
X
XML-RPC (Extensible Markup Language-Remote Procedure Call), 132
Y
yourself, scaling
influencing cost, 354–355
influencing schedule, 355–357
influencing scope, 350–354
overtime and, 347–349
overview of, 347
self-management, 349–350
your own impact, 349
Z
zero-downtime
backups, 159
updates, 139