From the link you provided:
"The primary reason for the deficit falling by $1.7 trillion under Biden was that the deficit had skyrocketed to a record high of about $3.1 trillion in fiscal 2020 during the early stage of the pandemic under then-President Donald Trump."
https://www.cnn.com/2023/03/16/politics/fact-check-biden-deficit-minimum-tax
So yes, the deficit declined over certain years of the Biden fiasco. (You may be confusing deficit with debt. Deficit is an annual number, debt is the cumulative aggregate of multiple deficits.)