(CNN) - A phone call demanding an immediate payment to the IRS sounds pretty scary. That's likely why it's a favorite among scammers. The scammers typically target the elderly and new immigrants, threatening arrest or the loss of a driver's license if they don't pay up. The fake phone calls top the IRS's annual list of dirty dozen tax scams. The government says nearly 3,000 victims have lost $14 million so far. Other scammers say you've got a refund, and need to hand over your personal data to claim it. The IRS says if it needs to contact taxpayers, it's usually by mail first, not over the phone.