improbable units
When you compute the Poisson-distributed probability of observing $k$ events in some time interval, you find
$$\begin{aligned} P(k) &= \frac{e^{-λ} λ^k}{k!} \end{aligned}$$
where $λ$ is the mean number of observed events. Here $λ$ needs to be dimensionless, because it's in the argument to the exponent and it's being raise to some power. But if you graduate to the exponential distribution, which describes the waiting time between successive events, then $λ$ acquires units of inverse time:
$$\begin{aligned} \mathrm dP(t) &= λ e^{-λ t} \mathrm dt \end{aligned}$$
Things that magically change units bother me. What's happening here?
I suspect that it's related to some integral which connects the continuous and discrete distributions. Let's imagine a scenario where we have a mean time $\tau_s = 1\text{ second}$ between events. The expected number of events per second is $λ_s = \tau_s^{-1} = 1\,\mathrm s^{-1}$, while the expected number of events per hour is $λ_h = 60\,\mathrm h^{-1}$. We have to get the same number of expected events in any interval:
$$\begin{aligned} \int_0^{(T)\,\mathrm s} λ_s e^{-λ_s t} \mathrm dt &= \int_0^{(T)\,\mathrm h} λ_h e^{-λ_h t} \mathrm dt \\ 1-e^{-λ_s \cdot (T)\,\mathrm s} &= 1-e^{-λ_h \cdot (T)\,\mathrm h} \end{aligned}$$
Let's get rid of $\lambda$, since it means different things in different contexts. We have conventionally $\tau = 1/\lambda$ for the mean time between events. Let's define $\beta = T/\tau$ as the mean number of events expected during some interval $T$.
$$\begin{aligned} \text{discrete:} && P(k) &= \frac{e^{-\beta} \beta^k}{k!} \\ \text{continuous:} && \mathrm dP(t) &= \frac{1}{\tau} e^{- t/\tau} \mathrm dt \end{aligned}$$
Now we should be able to recover one of these distributions from the other one somehow. The discrete probability should be something like
$$\begin{aligned} P\left(K=k \middle| t = \beta\tau\right) \end{aligned}$$
The probability for no events is
$$\begin{aligned} P(t > \tau) &= \int_{\beta\tau}^\infty \mathrm dP(t) &&= e^{-\beta} \end{aligned}$$
which matches up. I think that solves the unit problem.
How do I find the probability for a single event? I'm expecting it to be $\beta e^{-\beta}$. I can think of a few ways to do it, and I'm kind of suspecting that it might be in my upcoming reading — especially since the text talks about how the gamma distribution is a multi-event extension of the exponential distribution.
[thirty minutes later:] It was. The text goes in the opposite direction, deriving the continuous distributions from the discrete one. I guess one way to think about it is to remember having derived the Poisson distribution from the binomial distribution in the limit of enormously many things.
In terms of "what's the fundamental starting point for probability", the binomial distribution looks like the leading candidate.