Pointing to http://www.octave.org is good, because that gives people a direct way to find out more. If citation of a URL is not allowed by a publisher, or if you also want to point to a traditional reference, then you can cite the Octave manual:
@BOOK{eaton:2008,
author = "John W. Eaton and David Bateman and Søren Hauberg",
title = "GNU Octave Manual Version 3",
publisher = "Network Theory Limited",
year = "2008",
isbn = "0-9546120-6-X"
}