A smart contract is a small script that is directly stored and executed on the blockchain by bakers for Tezos (and miners in a POW setting).
Smart contracts enable users to handle tokens, derivative tokens (as in ERC20 contracts), conditions, events, etc., in an explicit and automatic way.
A 'dumb' contract is a regular contract written on a piece of paper without the automatic execution.
Also smart contracts need to be fully explicit whereas regular contracts leave a lot to interpretation.
A nice primer by the CFTC, a US regulator: