Starting April 21 Google will prioritize ranking for websites that have taken mobile devices seriously, and will lower ranking for sites that haven't.
Sites that are mobile friendly and responsive will score higher than their competitors. You can test your site here, the results look like this:
So what can you do? Well this entirely depends on your website platform. If you're using one of the giants like WordPress, Drupal or Joomla, you may just need to install an updated template that does all the mobile things automatically. If your entire site is hand-built or generated by a discount "1-click website builder" tool, you might have a lot of work ahead of you to make it mobile friendly.
Google has more details and agreat guide here: https://developers.google.com/webmasters/mobile-sites/