popd

TLDR

Remove the top directory from the stack and cd to it

$ popd

Remove the Nth directory (starting from zero to the left from the list printed with dirs)

$ popd +N

Remove the Nth directory (starting from zero to the right from the list printed with dirs)

$ popd -N