Sure. Have a look at Snips [0], Mycroft [1], Jasper [2] or Adrian [3] which are all open source home assistant devices with voice activation, although some do use Googles NLP cloud APIs for speech processing. Snips seems the most interesting, but looks terrible - a bare PCB on your wall? Really?
0. https://snips.ai/
1. https://mycroft.ai/
2. http://jasperproject.github.io/
3. http://www.theadrianproject.com/