\documentclass[11pt]{article}
\usepackage{amsmath,amssymb,amsthm,graphicx}
\usepackage{fullpage}
\usepackage[capitalise,nameinlink]{cleveref}
\crefname{lemma}{Lemma}{Lemmas}
\crefname{fact}{Fact}{Facts}
\crefname{theorem}{Theorem}{Theorems}
\crefname{corollary}{Corollary}{Corollaries}
\crefname{claim}{Claim}{Claims}
\crefname{example}{Example}{Examples}
\crefname{problem}{Problem}{Problems}
\crefname{setting}{Setting}{Settings}
\crefname{definition}{Definition}{Definitions}
\crefname{assumption}{Assumption}{Assumptions}
\crefname{subsection}{Subsection}{Subsections}
\crefname{section}{Section}{Sections}

\DeclareMathOperator*{\E}{\mathbb{E}}
\let\Pr\relax
\DeclareMathOperator*{\Pr}{\mathbb{P}}

\newcommand{\eps}{\varepsilon}
\newcommand{\inprod}[1]{\left\langle #1 \right\rangle}
\newcommand{\R}{\mathbb{R}}

\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {\bf CS 270: Combinatorial Algorithms and Data Structures
        } \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{Scribe: #4}{Lecture #1}}

\newtheorem{theorem}{Theorem}[section]
\newtheorem*{theorem*}{Theorem}
\newtheorem{itheorem}{Theorem}

\newtheorem{subclaim}{Claim}[theorem]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem*{proposition*}{Proposition}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem*{lemma*}{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem*{conjecture*}{Conjecture}
\newtheorem{fact}[theorem]{Fact}
\newtheorem*{fact*}{Fact}
\newtheorem{exercise}[theorem]{Exercise}
\newtheorem*{exercise*}{Exercise}
\newtheorem{hypothesis}[theorem]{Hypothesis}
\newtheorem*{hypothesis*}{Hypothesis}
\newtheorem{conjecture}[theorem]{Conjecture}

\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{setting}[theorem]{Setting}
\newtheorem{construction}[theorem]{Construction}
\newtheorem{example}[theorem]{Example}
\newtheorem{question}[theorem]{Question}
\newtheorem{openquestion}[theorem]{Open Question}
% \newtheorem{algorithm}[theorem]{Algorithm}
\newtheorem{problem}[theorem]{Problem}
\newtheorem{protocol}[theorem]{Protocol}
\newtheorem{assumption}[theorem]{Assumption}
\newtheorem{exercise-easy}[theorem]{Exercise}
\newtheorem{exercise-med}[theorem]{Exercise}
\newtheorem{exercise-hard}[theorem]{Exercise$^\star$}
\newtheorem{claim}[theorem]{Claim}
\newtheorem*{claim*}{Claim}

\newtheorem{remark}[theorem]{Remark}
\newtheorem*{remark*}{Remark}
\newtheorem{observation}[theorem]{Observation}
\newtheorem*{observation*}{Observation}


% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
% \topmargin 0pt
% \advance \topmargin by -\headheight
% \advance \topmargin by -\headsep
% \textheight 8.9in
% \oddsidemargin 0pt
% \evensidemargin \oddsidemargin
% \marginparwidth 0.5in
% \textwidth 6.5in

% \parindent 0in
% \parskip 1.5ex

\usepackage{amsfonts, dsfont}

\begin{document}

\lecture{26 --- April 27, 2023}{Spring 2023}{Prof.\ Jelani Nelson}{Jelani Nelson}

% \documentclass[12pt]{article}

% \usepackage{amsmath,amssymb,amsthm}
% \usepackage{graphicx}

% \begin{document}

% \title{Chronogram method for partial sums, and more}

% \maketitle

\section{Lower bounds and the cell probe model}

Typically in this class we have assumed the {\em word RAM} model of computation. That is, our machine has some constant number of registers and runs a program stored in memory. The first instruction of the program is stored at some particular memory address $M_0$, the next one at $M_1$, etc.\ (and some instructions can be \texttt{JUMP}s, which set the program counter to some other value). These instructions all operator on {\em words}. A word is a basic unit of storage, which we assume is $w$ bits. For example, the \texttt{ADD} instruction can add two words at a time (modulo $2^w$), and memory addresses are $w$ bits long (and hence the total memory of the machine is at most $2^w$), etc.

How do we prove lower bounds on data structures and algorithms in this model? One of the most robust ways is to only count \texttt{LOAD} and \texttt{STORE} instructions, since these are instructions that pretty much any real machine has, whereas some of the other instructions are not necessarily universal (e.g.\ \texttt{POPCOUNT} and \texttt{MostSignificantBit} instructions). That is, we only count memory reads and writes. This model of complexity, where all computation is free and only reads/writes from/to memory are counted, is known as the {\em cell probe model} of Yao \cite{Yao78}. In the remainder of this note, we will be proving lower bounds in this model.


\section{Partial sums lower bound}

Consider the dynamic partial sums problem over some group $G$ with operation `$+$' on an $n$-dimensional array $A$. Initially the array has all entries set to $0$ (the identity element of the group).

\begin{itemize}
\item \texttt{update}$(i, b)$: $A\text{[}i\text{]}\leftarrow b$
\item \texttt{query}$(i)$: return $\sum_{j=1}^iA\text{[}j\text{]}$
\end{itemize}

Working with a word of size $w$, it is natural to consider $G$ to be the cyclic group $\mathbb{Z}_{2^w}$. There is a data structure solving this problem with $O(\lg n)$ query time and update time, by building a complete binary search tree with $[n]$ as leaves. For a node $u$ a group element $g_u$ is stored, maintaining the invariant that the answer to \texttt{query}$(i)$ is the sum of all node-associated group elements on the root-to-leaf path to leaf $i$. This invariant can be maintained with $O(\log n)$ update and query. Note that the group does not {\em have} to be $\mathbb{Z}_{2^w}$. Below we show a lower bound for an even simpler problem: where the group is $\mathbb{Z}_2$ (note this problem is easier, since it is equivalent to saying we only want to find the least significant bit of the answer for $G = \mathbb Z_{2^w}$).

We show a lower bound for dynamic partial sums over $\mathbb Z_2$ due to Fredman and Saks \cite{FredmanS89} of $t_q = \Omega(\lg n / \lg(t_u w))$, where $t_u$ is the update time and $t_u$ is the query time. In particular, this implies $\max\{t_u, t_q\} = \Omega(\lg n / \lg\lg n)$. The optimal lower bound of $\max\{t_u, t_q\} = \Omega(\lg n)$ was not shown until 15 years later by P\v{a}tra\c{s}cu and Demaine \cite{PatrascuD04} --- we will not show that today.

The lower bound works as follows, and is known as the {\em chronogram technique}. The way we describe the chronogram technique will be in the family of what we call encoding techniques. Essentially what we say is that if we have a data structures with a bound $t_q$ that is too small, then we could use that data structure an an encoding scheme to compress elements of some set $S$ into $\ll \lg|S|$ bits. Clearly this is impossible by the pigeonhole principle, so $t_q$ must be large. We now give the details. Almost all of my understanding of how this works is due to a conversation with Kasper Green Larsen. In particular, the presentation of the chronogram technique below {\em is} slightly different than both the original \cite{FredmanS89} and the treatment in Miltersen's survey \cite{Miltersen99}. I personally find the explanation given below slightly more intuitive.

Consider a data structure $\mathcal D$ that works on operation sequences that look as follows. The operation sequence has $n$ updates, followed by one uniformly random query. We group these $n$ updates together into what we call {\em epochs}. Epoch $1$ is the last epoch of updates (right before the query), and epoch $2$ comes right before it, etc. Epoch $i$ will be a sequence of $\beta^i$ updates for some $\beta$ we will choose later. Thus the number of epochs is $\lg_{\beta} n$.

Recall that there is an $n$-dimensional array $A$ being updated in dynamic partial sums. In the updates of epoch $i$, we update the set of array entries with index of the form $j\cdot (n/\beta^i)$ for $j=1,\ldots,\beta^i$. The entries are assigned independent uniform random bit values. We use $b_{i,1},...,b_{i,\beta^i}$ to denote these random bit values, where $b_{i,j}$ is the value assigned to the entry $j\cdot (n/\beta^i)$ during the updates of epoch $i$.

At the end of epoch $1$, we execute a single uniformly random partial sum query, i.e.\ \texttt{query}$(k)$ for a uniformly random $k\in[n]$.

Now, we color the memory cells of the data structure (there are at most $2^w$ memory cells) after the entire sequence of updates has been processed. A memory cell $c$ is colored with color $i$ if its contents were changed during epoch $i$ but not in any epoch $j<i$. That is, $i$ is the {\em last} epoch in which $c$ was updated. Let $C_i$ denote the set of cells colored with color $i$. If the worst case update time is $t_u$, then clearly $|C_i| \le  t_u \beta^i$ for all $i$.

Let $T$ denote the (random) set of memory cells probed by the uniformly random query after the $n$ updates. Our goal is to show that:

$$
\forall i,\ \E|T \cap C_i| = \Omega(1) .
$$
Since the $C_i$ are disjoint, we get from linearity of expectation that 
$$
\E |T| \ge \sum_{i=1}^{\lg_\beta n} \E|T \cap C_i| ,
$$
and a lower bound of $t_q = \Omega(\log_{\beta} n)$ would follow.

To prove this, assume for contradiction that there exists some $i$ such that $\E|T \cap C_i| \le \alpha$ for some very small constant $\alpha$ to be determined. We will use this assumption to create an impossible encoding of $b_{i,1},...,b_{i,\beta^i}$. Notice that the update values for epoch $i$ are independent of epochs $j \neq i$. Thus $H(b_{i,1}...b_{i,\beta^i} | (b_{i',\cdot})_{i'\neq i}) = \beta^i$. This means that an encoder and a decoder can share knowledge of all updates in all epochs other than $i$, and we would still have to write down $\beta^i$ bits in an encoding. In addition to this, let $R_1,\ldots,R_{\beta^i}$ be a sequence of random variables such that $R_j$ gives a uniform random index between $j\cdot (n/\beta^i)$ and $(j+1)\cdot (n/\beta^i)-1$.  Clearly we can also condition on these as they are independent of the update bits. The encoding argument now goes as follows:


We examine $R_1,\ldots,R_{\beta^i}$ and run the corresponding queries on $\mathcal D$. For each query, we check whether it reads a cell in $C_i$.  If we pick a uniform random $R_j$, then the distribution of the query is uniform random. This means that $\E_{j,R_1,\ldots,R_{\beta^i}} |T(R_j) \cap C_i|\le \alpha$ (here $T(R_j)$ is the set of cells probed by the query $R_j$). Thus
$$
\E_{R_1,\ldots,R_{\beta^i}} \sum_{j=1}^{\beta^i} |T(R_j)\cap C_i| \le \alpha \beta^i .
$$
Thus by Markov's inequality and a union bound, we get that with probability $3/4$, the number of queries $R_j$ which read in $C_i$ is upper bounded by $4\alpha\beta^i < \beta^i/100$ for $\alpha = 1/400$. If this is not the case, our encoding consists of a $0$-bit, follows by a naive encoding of $b_{i,1},\ldots,b_{i,\beta^i}$, costing $\beta^i+1$ bits. 

Otherwise, we start by writing a $1$-bit followed by a description of which queries amongst $R_1,\ldots,R_{\beta^i}$ read inside $C_i$. For those queries, we also write down their answer. This costs $\lg\binom{\beta^i}{\beta^i/100} + \beta^i/100 < \beta^i/4$ bits. Next, we write down all memory cells in $C_{i-1},\ldots,C_{1}$. This costs $\sum_{j<i} O(|C_j| w) = O(\beta^{i-1}t_u w)$ bits. If we choose $\beta = (t_u w)^2$, this is $o(\beta^i)$ bits.


The expected size of the encoding is less than $1+(3/4)(\beta^i/4+ o(\beta^i))(3/4) + (1/4)\beta^i$ bits, thus by Shannon's source coding theorem, the entropy of the encoded message, $M$, is $H(M) < \beta^i/2$ bits.

\begin{figure}
\begin{center}
\scalebox{0.678}{\includegraphics{yellow.pdf}}
\end{center}
\caption{Above is the array $A$, and its entries are broken into contiguous blocks of size $n/\beta^i$. $R_j$ is then chosen to be a uniformly random index in the $j$th block, i.e.\ $R_j$ is chosen uniformly at random in the interval $[j\cdot (n/\beta^i), (j+1)\cdot (n/\beta^i))$. The entries in $A$ colored in yellow correspond to the entries updated in epoch $i$.}\label{fig:epochs}
\end{figure}

To derive the contradiction, we need to show that this encoded information is enough to (almost) fully recover $(b_{i,j})_{j=1}^{\beta^i}$. We will show that a $(1-\beta)$-fraction of the bits $b_{i,1},\ldots,b_{i,\beta^i}$ are completely determined from $M$: If we wrote a $0$-bit first, this follows trivially (in fact then all the bits are recovered). If we write a $1$-bit first, we first obtain the set of queries that read inside $C_i$ and their corresponding answers. From this, we can also deduce the queries that do not read inside $C_i$. Since the epochs $j\neq i$ are known to the decoder, he can execute the update algorithm of $\mathcal D$ on all updates preceeding epoch $i$. Now, since he knows which queries do not read in $C_i$, he can run $\mathcal D$'s query procedure on those queries. Whenever $\mathcal D$'s query procedure requests a memory cell, he checks whether the cell is included in the sets $C_{i-1},\ldots,C_{1}$ (which is in the encoding). If so, he has the contents and can continue.  Otherwise, he knows they are not in sets $C_i,...,C_1$ and therefore the contents must be the same as they were just before epoch $i$. But these contents are known because he just ran all updates preceeding epoch $i$. He can thus finish the query algorithm and now has the answer to every query $R_1,\ldots,R_{\beta^i}$. 

Now, let us remember which cells are updated in epoch $i$ (the yellow cells in Figure~\ref{fig:epochs}). By the end of epoch $i$, the first yellow cell contains the value $b_{i,1}$, the second yellow cell contains $b_{i,2}$, etc. All the cells which are not yellow in between were written in previous epochs, and thus the decoder knows them. Therefore, by knowing all the answers to the $R_1,\ldots,R_{\beta^i}$, the decoder can recover all the bits written in the yellow positions (since the answer to query $R_j$ is the XOR of all bits up to and including $R_j$, and the decoder knows all of those bits except for one, the one in yellow!). There is a slight catch: epochs $i-1,i-2,\ldots 1$ do overwrite some of the yellow cells with new bits, and thus for the blocks $j$ where this overwriting happens, the decoder will not learn $b_{i,j}$ from the answers to the queries $R_1,\ldots,R_j$. However, note that the only indices that are overwritten in future epochs are the indices $0, n/\beta^{i-1}, 2n/\beta^{i-2}, \ldots = 0, \beta n/\beta^i, 2\beta n/ \beta^i, \ldots$. In other words, only a $\beta$-fraction are overwritten, so the decoder does learn the $(1-\beta)$-fraction of the bits that weren't overwritten. But these are still $(1-\beta)\beta^i$ uniformly random bits, and any encoding to recover them should have expected encoding length at least $(1-\beta)\beta^i$, but we are achieving total expected encoding length $\E |M| < \beta^i/2$, which is still a contradiction. In other words, one can view this entire scheme as a compression scheme not for {\em all} the bits $b_{i,1},\ldots,b_{i,\beta^i}$, but just the $(1-\beta)$-fraction that the decoder actually learns. Alternatively, the encoder can just always write $b_{i,1}, b_{i,\beta + 1},b_{i,2\beta + 1}, \ldots, b_{i, (\beta^{i-1}-1)\beta + 1}$ as part of the encoding as well, which only adds $\beta^{i-1} = o(\beta^i)$ to the encoding length, so that the decoder can in fact learn all the bits at the end of the day.

\section{Lower bound for dynamic connectivity}

Given the lower bound of $\max\{t_u, t_q\} = \Omega(\lg n / \lg\lg n)$ of the previous section for dynamic partial sums over $\mathbb Z_2$, we can obtain the same lower bound for dynamic connectivity via a reduction due to \cite{MiltersenSVT94}. In fact the reduction holds even if the underlying graph is promised to always be a collection of two vertex-disjoint trees, which implies optimality of the link-cut trees of Sleator and Tarjan \cite{SleatorT83}. In dynamic connectivity we wish to support the following operations, where the graph $G$ starts as the empty graph on $n$ vertices:

\begin{itemize}
\item \texttt{insert}$(u, v)$: insert the edge $(u, v)$ into $G$
\item \texttt{delete}$(u, v)$: delete the edge $(u, v)$ from $G$, if it exists
\item \texttt{connected}$(u, v)$: return \texttt{True} if $u, v$ are in the same connected component, and false otherwise
\end{itemize}

The reduction works as follows. Recall we want to solve dynamic partial sums over $\mathbb Z_2$, where there is some underlying array $A\text{[}0\ldots n-1\text{]}$ and the answer to \texttt{query}$(i)$ is $A\text{[}0\text{]}\oplus A\text{[}1\text{]}\oplus \cdots \oplus A\text{[}i\text{]}$. We maintain a graph $G$ on vertex set $[2n+3]$ with the following edge set $E$:

$$
E = \left(\bigcup_{i : A\text{[}i\text{]} = 0} \{(2i+1, 2i+3), (2i+2, 2i+4)\right) \cup \left(\bigcup_{i : A\text{[}i\text{]} = 1} \{(2i+1, 2i+4), (2i+2, 2i+3) \}\right) .
$$

\begin{figure}
\begin{center}
\scalebox{0.5}{\includegraphics{conn_reduction.pdf}}
\end{center}
\caption{Example reduction from dynamic partial sums over $\mathbb Z_2$ to dynamic connectivity. Numbers in the top are vertex ID's. Red edges correspond to $1$'s in the array $A$, and black edges are $0$'s. The underlying array in this example is $A\text{[}0\ldots 6\text{]} = \text{[}1, 0, 0, 1, 1, 1, 0\text{]}$. Thus $n = 7$, so the total number of vertices in $G$ is $2n + 3 = 17$. The blue dashed line is inserted before querying \texttt{connected}$(1, 17)$ to test whether $A\text{[}0\text{]}\oplus A\text{[}1\text{]}\oplus A\text{[}2\text{]}\oplus A\text{[}3\text{]} = 0$.}\label{fig:reduction}
\end{figure}

Note that a change to some entry $A\text{[}i\text{]}$ corresponds to deleting and inserting $O(1)$ new edges in $G$ (to be precise: two edge removals and two edge insertions into $G$). One can show that $A\text{[}0\text{]}\oplus A\text{[}1\text{]}\oplus \cdots \oplus A\text{[}i\text{]}$ equals $0$ iff \texttt{connected}$(1, 2n+3)$ is \texttt{True} after inserting the edge $(2i+,3, 2n+3)$ into $G$. Thus \texttt{query} in partial sums can be implemented by one edge insertion, one \texttt{connected} query, then one edge deletion in $G$. Thus the maximum operation time in dynamic connectivity must be at least that for partial sums over $\mathbb Z_2$, which is $\Omega(\lg n / \lg\lg n)$ as shown in the last section.

The intuition behind this reduction is the following. If all array entries $A$ had been zero, then $E$ would be $\{(1, 3), (3, 5), (5, 7) \ldots\} \cup \{(2, 4), (4, 6), (6, 8), \ldots \}$. That is, there would be an ``even path'' (only touching vertices with even ID's) and an ``odd path'' (only touching vertices with odd ID's), such that after every index $i$ in $A$, the even path is always one step ahead of the odd path. These are the black edges in Figure~\ref{fig:reduction}. Whenever there is a $1$ at some entry in $A$, it introduces the red edges into $G$ which swap the even and odd paths. That is, it makes the even path the odd path, and the odd path the even path. Note that $1$ and $2n+3$ are both odd numbers, so had all entries of $A$ been $0$, they would both be in the odd path together. However, due to the paths swapping at $1$'s, we essentially want to check that the number of swaps from the beginning up to the edges added by $A\text{[}i\text{]}$ is even.

\section{What's happened since?}
\begin{itemize}
\item $\max\{t_q, t_u\} = \Omega(\lg n)$ for both dynamic connectivity and partial sums (over $\mathbb Z_{2^w}$) \cite{PatrascuD04}.
\item $\max\{t_q, t_u\} = \Omega((\lg n/\lg\lg n)^2)$ for dynamic weighted orthogonal range counting in $2D$ , but only when the weights are $\lg^{2+\epsilon} n$-bit integers \cite{Patrascu07}.
\item $t_q = \Omega((\lg n / \lg(w t_u))^2)$ for dynamic range counting in $2D$ with $\lg n$-bit weights \cite{Larsen12}.
\item $\max\{t_u, t_q\} = \tilde{\Omega}(\lg^{1.5} n)$ for parity dynamic range counting in $2D$ \cite{LarsenWY17}.
\item Many lower bounds known for other problems, e.g.\ union-find, predecessor, dynamic min spanning forest, planarity testing, dynamic marked ancestor, several computational geometry problems, approximate nearest neighbor in $\ell_p$ for various $p$, etc. See \cite{Patrascu11} and also the older survey of Miltersen \cite{Miltersen99}.
\end{itemize}

\newcommand{\etalchar}[1]{$^{#1}$}
\begin{thebibliography}{FS89}

\bibitem[FS89]{FredmanS89}
Michael~L.~Fredman, Michael~E.~Saks.
\newblock The Cell Probe Complexity of Dynamic Data Structures.
\newblock {\em STOC}, pages 345--354, 1989.

\bibitem[L12]{Larsen12}
Kasper~Green~Larsen.
\newblock The cell probe complexity of dynamic range counting. 
\newblock {\em STOC}, pages 85--94, 2012.

\bibitem[LWY17]{LarsenWY17}
Kasper~Green~Larsen, Omri~Weinstein, Huacheng~Yu.
\newblock Crossing the Logarithmic Barrier for Dynamic Boolean Data Structure Lower Bounds.
\newblock {\em CoRR} abs/1703.03575, 2017.

\bibitem[M99]{Miltersen99}
Peter~Bro~Miltersen. 
\newblock Cell probe complexity --- a survey.
\newblock {\em In 19th Conference on the Foundations of Software Technology and Theoretical Computer Science (FSTTCS)}, 1999. Advances in Data Structures Workshop.

\bibitem[MSVT94]{MiltersenSVT94}
Peter~Bro~Miltersen, Sairam~Subramanian, Jeffrey~Scott~Vitter, Roberto~Tamassia.
\newblock Complexity Models for Incremental Computation.
\newblock {\em Theor. Comput. Sci.} 130(1), pages 203--236, 1994.

\bibitem[P07]{Patrascu07}
Mihai~P\v{a}tra\c{s}cu.
\newblock Lower bounds for $2$-dimensional range counting.
\newblock {\em STOC}, pages 40--46, 2007.

\bibitem[P11]{Patrascu11}
Mihai~P\v{a}tra\c{s}cu.
\newblock Unifying the Landscape of Cell-Probe Lower Bounds.
\newblock {\em SIAM J. Comput.} 40(3), pages 827--847, 2011.

\bibitem[PD04]{PatrascuD04}
Mihai~P\v{a}tra\c{s}cu, Erik~D.~Demaine.
\newblock Tight bounds for the partial-sums problem.
\newblock {\em SODA}, pages 20--29, 2004.

\bibitem[ST83]{SleatorT83}
Daniel~Dominic~Sleator, Robert~Endre~Tarjan.
\newblock A Data Structure for Dynamic Trees.
\newblock {\em J. Comput. Syst. Sci.} 26(3), pages 362--391, 1983.

\bibitem[Y78]{Yao78}
Andrew~Chi-Chih~Yao.
\newblock Should Tables Be Sorted?
\newblock {\em FOCS}, pages 22--27, 1978.

\end{thebibliography}

\end{document}