Skip to content

Instantly share code, notes, and snippets.

@Nikolaj-K
Created January 9, 2026 17:46
Show Gist options
  • Select an option

  • Save Nikolaj-K/ab57432b2a47b02c57bd77797083b237 to your computer and use it in GitHub Desktop.

Select an option

Save Nikolaj-K/ab57432b2a47b02c57bd77797083b237 to your computer and use it in GitHub Desktop.
Quickie on LogSumExp vs Max

Video where this script is discussed: https://youtu.be/Lsf4eAGvODs

Consider a non-strictly ordered space $X$ with an (absolutely homogenous) absolute value function $|\cdot|: X\to X$ If $B:(X\times X)\to X$ is a symmetric map, then

$B(\max(x, y)-x, \max(x, y)-y) = B(0, |x-y|)$

Proof

With $|-a| = |a|$ and $B(a,b)=B(b,y)$, the equation is symmetric in $x$ and $y$. So w.l.o.g. sconsider $x\ge y$ so that both sides read $B(0, x-y)$.

Proof (genealized)

We get a generalization for any two group elements $x,y\in G$ and "flag" depending on $P$,

$f_P( x, y) := \begin{cases} x & \mathrm{if}\ P\\ y & \mathrm{else} \end{cases}$

With case analysis on $P$, we find

${f_P( x, y)\cdot x^{-1}, f_P( x, y)\cdot y^{-1}} = {e, (x\cdot y^{-1})^{f_P( 1, -1)}}$

In a ${\mathbb Z}$-module, the last term reads $f_P( 1, -1)\cdot(x-y)$.

Going back, if $G$ is equipped with a non-strict partial order, then $f_{x\ge y}(x, y) = \max(x, y)$. If there's further an absolutely homogeneous (we use norm notation here, which has this), then

${\max(x, y)-x, \max(x, y)-y} = {0, |x-y|}$

and by similar considerations $f_{x\le y}(x, y) = \min(x, y)$ and

${\min(x, y)-x, \min(x, y)-y} = {0, -|x-y|}$

Applications

Consider now the reals, just to make the known formulas more recognizable. Write $m_{x,y}$ for $\max(x, y)$ or $\min(x, y)$ so that $B(m_{x,y}-x, m_{x,y}-y) = B(0, \pm|x-y|)$ Reminder: The case with $+$ corresponds to $\max$.

  • Consider plain addition, $B:=+$

We get $2m_{x,y}-x-y = \pm|x-y|$ or $m_{x,y} = \dfrac{x+y\pm|x-y|}{2}$

  • Consider plain multiplication, $B:=\cdot$

We get $m_{x,y}^2 - (x+y)\cdot m_{x,y} + x\cdot y = 0$ or, as above, $m_{x,y} = \dfrac{x+y\pm \sqrt{(x-y)^2}}{2}$

  • Consdier $B(x_1, x_2) := \mathrm{SumExp}(-\langle x_1, x_2\rangle^T)$ where $\mathrm{SumExp}_b({\vec x}) := \sum_i b^{x_i}$,

Also let $\mathrm{LogSumExp}_b := \log_b\circ,\mathrm{SumExp}_b$ and remember $b^0 = 1$ and note that $\mathrm{SumExp}_b({\vec x}+c\cdot {\vec 1}) = \mathrm{SumExp}_b({\vec x})\cdot b^c$.

We get $\mathrm{SumExp}b(\langle x, y\rangle\rangle) / b^{ m{x,y}} = 1 + b^{\mp|x-y|}$

And so e.g., for the

$\mathrm{LogSumExp}_{\mathrm e}(\langle x, y\rangle) = \max(x, y) + \ln\left(1 + {\mathrm e}^{-|x-y|}\right)$

import math b=5; x = 1.2; y=3.4 num = b**x + b**y # 244.85539064679085 denom = 1 + b ** -abs(x-y) # 1.0289911865471078 math.log(num/denom, b) # 3.4000000000000004 denom = 1 + b ** +abs(x-y) # 35.493241536530384 math.log(num/denom, b) # 1.1999999999999997
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment