The beans manual
Table of Contents
1 About beans
beans is a plain text accounting tool in the tradition of ledger, hledger and beancount.
1.1 Project goals
1.1.1 Built-in file importers
The greatest pain of plain text accounting is to import financial
statements from financial service providers such as banks and credit
cards. beans strives to include importers for known formats.
1.1.2 Automatic categorization
beans provides a Bayesien inference (machine learning) engine to
automatically assign accounts based on your existing ledger.
1.1.3 beans always balances
beans aims to make it simple to do at-market valuations of your
assets, while maintaining correctness of the balance. Instead of
converting aggregated account data and adding valuation as an
afterthought, beans applies valuation directly at the transaction
level. For example, if you choose to display a balance at market
prices, beans will transparently insert valuation transactions to
reflect gains and losses due to price changes.
1.1.4 Transcode to beancount
Translate your beans journal into a beancount-compatible format to make use of the great tooling universe of beancount, such as the Fava web interface.
1.1.5 Simple implementation
beans is written in Haskell, an excellent language for concise and
beautiful code. Also, while beans adopts concepts from other tools,
compatiblity is not an objective.
1.2 Changes (if you used a previous version)
- The previous rules engine got replaced by a Bayesian inference mechanism, which in practice is much less work.
- The file format changed: no more indentation, for friendlier editing
- Multi-column reports
- Some more importers
- Lots of changes under the hood
2 Getting beans
2.1 Obtaining the source
beans' source code can be found on github. For the moment, you have
to build from source.
Make sure you have git installed and run:
git clone https://github.com/sboehler/beans.git
2.2 Installing beans
2.2.1 Using Stack
stack is probably the easiest option on most platforms to get
started quickly. First, follow the instructions here to install stack
for your platform. Then:
cd beans stack install
This will install the beans binary in $HOME/.local/bin/beans. Add this
directory to your path and you're good to go:
export PATH=~/.local/bin:$PATH
Please file a bug if you encounter problems using stack.
2.2.2 Using Nix
Make sure you have the nix package manager installed for your platform. Then:
cd beans nix-build --attr beans release.nix
The beans binary will be made available locally in ./result/bin/beans
and can be run from there. I am using nix to develop beans, so this
option is the one that is supported best.
2.2.3 Using Cabal
If you intend to develop beans or run the test suite. Make sure cabal and hpack are installed:
cd beans cabal new-build
The beans binary will be made available in ./dist/build/beans/beans and can be run from there.
2.3 Getting help
This guide is incomplete and might be out of date. To get an overview
of your possibilities, beans has a built-in help command:
beans --help
beans Usage: beans COMMAND A plain text accounting tool Available options: -h,--help Show this help text Available commands: balance Print a generic balance fetch Fetch latest prices import Import transactions infer Infer accounts transcode Transcode to beancount
3 Working with beans
beans is a command line tool and has some self-documenting
features. Run the following command to get started:
beans --help
beans Usage: beans COMMAND A plain text accounting tool Available options: -h,--help Show this help text Available commands: balance Print a generic balance fetch Fetch latest prices import Import transactions infer Infer accounts transcode Transcode to beancount
Use beans <command> --help to get further help on available options.
3.1 Importing data
beans offers assistance with importing bank account statements, the most tiring process in plain text accounting.
3.1.1 Importers
In a first step, bank account statements need to be parsed. The following is what a statement from my bank might look like.
Datum von:;2017-05-23 Buchungsart:;Alle Buchungen Konto:;CHXXXXXXXXXXXXXXXXXXX Währung:;CHF Buchungsdatum;Avisierungstext;Gutschrift;Lastschrift;Valuta;Saldo 2017-06-01;"Acme Corp";;-135.00;2017-06-01; 2017-05-31;"FOOBAR COMMUNICATIONS";;-52.60;2017-05-31; 2017-05-29;"Transfer 01-45-06 Bigbank";;-1040.00;2017-05-29;36096.54 2017-05-26;"Cleaning Service";;-300.00;2017-05-24;37136.54 2017-05-26;"Supermarket Fine Foods";;-115.20;2017-05-26; 2017-05-26;"Bank Nextgen ";;-250.00;2017-05-26; 2017-05-26;"Tire Car Sales";;-95.85;2017-05-26; Disclaimer: Dies ist kein durch PostFinance AG erstelltes Dokument. PostFinance AG ist nicht verantwortlich für den Inhalt.
This is a kind of a CSV format, but then not really, so this is
difficult to parse with a general parser. beans strives to have
built-in support for as many file formats as possible. Currently, the
following parsers are available:
- us.interactivebrokers
- ch.postfinance
- ch.supercardplus
- ch.cumulus
- ch.swissquote
3.1.2 Importing data
If you are lucky enough that the format of your bank is supported, converting the bank account statement to transactions is a simple command:
beans import -i ch.postfinance -a Assets:Checking examples/postfinance.csv
import outputs a simple list of transactions to stdout, in a format
parseable by beans and with unknown accounts marked as TBD. It is up to the
user to pipe them into a journal or a temporary file for further
editing.
2017-05-26 "Cleaning Service" Assets:Checking -300 CHF TBD 300 CHF 2017-05-26 "Supermarket Fine Foods" Assets:Checking -115.2 CHF TBD 115.2 CHF 2017-05-26 "Bank Nextgen " Assets:Checking -250 CHF TBD 250 CHF 2017-05-26 "Tire Car Sales" Assets:Checking -95.85 CHF TBD 95.85 CHF 2017-05-29 "Transfer 01-45-06 Bigbank" Assets:Checking -1040 CHF TBD 1040 CHF 2017-05-31 "FOOBAR COMMUNICATIONS" Assets:Checking -52.6 CHF TBD 52.6 CHF 2017-06-01 "Acme Corp" Assets:Checking -135 CHF TBD 135 CHF
3.1.3 Assigning accounts
Previous version of beans used to feature a DSL to specify rules,
but this still involved too much thinking and writing.
beans now has a built-in Bayesien inference (a.k.a. machine
learning) tool. Run beans infer to automatically replace TBD
accounts in a target file based on the existing transactions in your
ledger. Sounds magic, but works very well in practice. You need to get
it started by assigning accounts manually for a few weeks.
3.2 Creating a journal
3.2.1 Journal files
Most commands in beans are based on a journal. beans journals uses a
format largely similar to beancount, with the exception that beans
neither requires nor allows indentation:
; -*- mode: Org; eval: (beancount-mode 1); -*- option "operating_currency" "USD" ** Accounts 2018-01-01 open Equity:Valuation 2018-01-01 open Equity:Val 2018-01-01 open Equity:OpeningBalance 2018-01-01 open Income:Salary 2018-01-01 open Expenses:Rent 2018-01-01 open Expenses:Groceries 2018-01-01 open Expenses:Books 2018-01-01 open Equity:Gains 2018-01-01 open Assets:Checking 2018-01-01 open Assets:Portfolio 2018-01-01 open Assets:Wallet ** Some Transactions *** Initial balances 2018-01-01 "Opening balance" Assets:Checking 15000 USD Equity:OpeningBalance *** Checking account 2018-01-02 "Rent January" Assets:Checking -2000 USD Expenses:Rent 2018-01-10 "Food" Assets:Checking -300 USD Expenses:Groceries 2018-01-25 "Salary January" Assets:Checking 4000 USD Income:Salary 2018-02-01 "Rent February" Assets:Checking -2000 USD Expenses:Rent 2018-01-12 "Hungry again" Assets:Checking -200 USD Expenses:Groceries 2018-01-20 "Something to read" Assets:Checking -100 USD Expenses:Books 2018-02-25 "Salary February" Assets:Checking 4000 USD Income:Salary 2018-03-01 "Rent March" Assets:Checking -2000 USD Expenses:Rent 2018-03-02 "Yet more food" Assets:Checking -200 USD Expenses:Groceries 2018-03-20 "More food" Assets:Checking -400 USD Expenses:Groceries 2018-03-25 "Salary March" Assets:Checking 4000 USD Income:Salary *** Investments 2018-01-10 "Investing in stock" Assets:Portfolio +10 AAPL Assets:Checking -1700 USD Equity:Gains ** Prices 2018-01-01 price BTC 13412 USD 2018-02-01 price BTC 9052 USD 2018-03-01 price BTC 10907 USD 2018-04-01 price BTC 6816 USD 2018-05-01 price BTC 9067 USD 2018-06-01 price BTC 7518 USD 2018-07-01 price BTC 6347 USD 2018-08-01 price BTC 7603 USD 2018-09-01 price BTC 7192 USD 2018-10-01 price BTC 6578 USD 2018-11-01 price BTC 6358 USD 2018-01-01 price AAPL 169 USD 2018-02-01 price AAPL 167 USD 2018-03-01 price AAPL 175 USD 2018-04-01 price AAPL 166 USD 2018-05-01 price AAPL 169 USD 2018-06-01 price AAPL 190 USD 2018-07-01 price AAPL 185 USD 2018-08-01 price AAPL 201 USD 2018-09-01 price AAPL 227 USD 2018-10-01 price AAPL 225 USD 2018-11-01 price AAPL 222 USD 2018-01-14 "I want to be part of this!" Assets:Wallet 0.25 BTC Assets:Checking -3500 USD Equity:Gains
I am using Emacs with org-mode to manage my journal in one large file
(org-mode titles are treated as comments), however there is an
include directive as well which allows for splitting a journal in
different files.
3.2.2 Conventions
- Accounts must be opened (and closed, if so desired) explicitly. They can be open for all commodities (default) or restricted to a list of them
- Account names must start with "Assets", "Liabilities", "Equity", "Expenses" or "Income", depending on their type.
- Transactions need not be ordered,
beansautomatically orders them by date. - Transactions with only one commodity can have a 'wildcard' booking, which is automatically assigned the total of the other bookings (this is the same as in other plain-text accounting tools)
- Unlike
beancount,beansuses the convention thatbalancedirectives refer to balances at the end of the given date. This is congruent with standard practices in the financial industry.
3.3 Balancing
3.3.1 Global balance
In addition to the predefined reports, beans has a generic balancing command:
beans balance -j examples/example.bean
This outputs a balance for all accounts without currency conversions, on the first and last day of the ledger
|------------------+------------+------------| | Account | 2018-01-01 | 2018-11-01 | |------------------+------------+------------| | Assets | | | | Checking | | | | USD | 15000.00 | 14600.00 | | Portfolio | | | | AAPL | | 10.00 | | Wallet | | | | BTC | | 0.25 | |------------------+------------+------------| | Equity | | | | Gains | | | | AAPL | | -10.00 | | BTC | | -0.25 | | USD | | 5200.00 | | OpeningBalance | | | | USD | -15000.00 | -15000.00 | |------------------+------------+------------| | Income | | | | Salary | | | | USD | | -12000.00 | |------------------+------------+------------| | Expenses | | | | Books | | | | USD | | 100.00 | | Groceries | | | | USD | | 1100.00 | | Rent | | | | USD | | 6000.00 | |------------------+------------+------------| | Total | | | | AAPL | | | | BTC | | | | USD | | | |------------------+------------+------------|
You want a multi-column balance?
beans balance -j examples/example.bean --interval quarterly
|------------------+------------+------------+------------+------------+------------| | Account | 2017-12-31 | 2018-03-31 | 2018-06-30 | 2018-09-30 | 2018-12-31 | |------------------+------------+------------+------------+------------+------------| | Assets | | | | | | | Checking | | | | | | | USD | | 14600.00 | 14600.00 | 14600.00 | 14600.00 | | Portfolio | | | | | | | AAPL | | 10.00 | 10.00 | 10.00 | 10.00 | | Wallet | | | | | | | BTC | | 0.25 | 0.25 | 0.25 | 0.25 | |------------------+------------+------------+------------+------------+------------| | Equity | | | | | | | Gains | | | | | | | AAPL | | -10.00 | -10.00 | -10.00 | -10.00 | | BTC | | -0.25 | -0.25 | -0.25 | -0.25 | | USD | | 5200.00 | 5200.00 | 5200.00 | 5200.00 | | OpeningBalance | | | | | | | USD | | -15000.00 | -15000.00 | -15000.00 | -15000.00 | |------------------+------------+------------+------------+------------+------------| | Income | | | | | | | Salary | | | | | | | USD | | -12000.00 | -12000.00 | -12000.00 | -12000.00 | |------------------+------------+------------+------------+------------+------------| | Expenses | | | | | | | Books | | | | | | | USD | | 100.00 | 100.00 | 100.00 | 100.00 | | Groceries | | | | | | | USD | | 1100.00 | 1100.00 | 1100.00 | 1100.00 | | Rent | | | | | | | USD | | 6000.00 | 6000.00 | 6000.00 | 6000.00 | |------------------+------------+------------+------------+------------+------------| | Total | | | | | | | AAPL | | | | | | | BTC | | | | | | | USD | | | | | | |------------------+------------+------------+------------+------------+------------|
3.3.2 Balancing options
- Flattening the account structure
There are some formatting options available:
beans balance -j examples/example.bean --format flat
|-----------------------+------------+------------| | Account | 2018-01-01 | 2018-11-01 | |-----------------------+------------+------------| | Assets:Checking | | | | USD | 15000.00 | 14600.00 | |-----------------------+------------+------------| | Assets:Portfolio | | | | AAPL | | 10.00 | |-----------------------+------------+------------| | Assets:Wallet | | | | BTC | | 0.25 | |-----------------------+------------+------------| | Equity:Gains | | | | AAPL | | -10.00 | | BTC | | -0.25 | | USD | | 5200.00 | |-----------------------+------------+------------| | Equity:OpeningBalance | | | | USD | -15000.00 | -15000.00 | |-----------------------+------------+------------| | Income:Salary | | | | USD | | -12000.00 | |-----------------------+------------+------------| | Expenses:Books | | | | USD | | 100.00 | |-----------------------+------------+------------| | Expenses:Groceries | | | | USD | | 1100.00 | |-----------------------+------------+------------| | Expenses:Rent | | | | USD | | 6000.00 | |-----------------------+------------+------------| | Total | | | | AAPL | | | | BTC | | | | USD | | | |-----------------------+------------+------------|
- Valuation at market prices
Valuation at market (i.e. latest available prices) is always an option:
beans balance -j examples/example.bean --val USD
|------------------+------------+------------| | Account | 2018-01-01 | 2018-11-01 | |------------------+------------+------------| | Assets | | | | Checking | 15000.00 | 14600.00 | | Portfolio | | 2220.00 | | Wallet | | 1589.50 | |------------------+------------+------------| | Equity | | | | Equity | | 1233.50 | | Gains | | 157.00 | | OpeningBalance | -15000.00 | -15000.00 | |------------------+------------+------------| | Income | | | | Salary | | -12000.00 | |------------------+------------+------------| | Expenses | | | | Books | | 100.00 | | Groceries | | 1100.00 | | Rent | | 6000.00 | |------------------+------------+------------| | Total | | | |------------------+------------+------------|
- Restricting dates and periods
We can also analyze changes in the balance between specific dates:
beans balance -j examples/example.bean --val USD --from 2018-06-30 --to 2018-10-30
|------------------+------------+------------| | Account | 2018-06-30 | 2018-10-30 | |------------------+------------+------------| | Assets | | | | Checking | 14600.00 | 14600.00 | | Portfolio | 1900.00 | 2250.00 | | Wallet | 1879.50 | 1644.50 | |------------------+------------+------------| | Equity | | | | Equity | 1263.50 | 1148.50 | | Gains | 157.00 | 157.00 | | OpeningBalance | -15000.00 | -15000.00 | |------------------+------------+------------| | Income | | | | Salary | -12000.00 | -12000.00 | |------------------+------------+------------| | Expenses | | | | Books | 100.00 | 100.00 | | Groceries | 1100.00 | 1100.00 | | Rent | 6000.00 | 6000.00 | |------------------+------------+------------| | Total | | | |------------------+------------+------------|
- Analyzing changes in valuation
Using filtering, we can eliminate the effect of income and expenses and track valuation changes over the entire period:
beans balance -j examples/example.bean --account-filter Equity --val USD
|------------------+------------+------------| | Account | 2018-01-01 | 2018-11-01 | |------------------+------------+------------| | Assets | | | | Checking | 15000.00 | 9800.00 | | Portfolio | | 2220.00 | | Wallet | | 1589.50 | |------------------+------------+------------| | Equity | | | | Equity | | 1233.50 | | Gains | | 157.00 | | OpeningBalance | -15000.00 | -15000.00 | |------------------+------------+------------| | Total | | | |------------------+------------+------------|
How about bitcoin?
beans balance -j examples/example.bean --val BTC
|------------------+------------+------------| | Account | 2018-01-01 | 2018-11-01 | |------------------+------------+------------| | Assets | | | | Checking | 1.12 | 2.30 | | Portfolio | | 0.35 | | Wallet | | 0.25 | |------------------+------------+------------| | Equity | | | | Equity | | -1.34 | | Gains | | 0.01 | | OpeningBalance | -1.12 | -1.12 | |------------------+------------+------------| | Income | | | | Salary | | -1.11 | |------------------+------------+------------| | Expenses | | | | Books | | 0.01 | | Groceries | | 0.09 | | Rent | | 0.55 | |------------------+------------+------------| | Total | 0.00 | 0.00 | |------------------+------------+------------|
beansderives arbitrary commodity conversions between all commodities, as long as there is at least one path (possibly over several links) from one commodity to another. For example, conversion from SomeStock in CHF is possible when prices of SomeStock in USD and CHF in USD is available. Inverted rates are derived automatically.At the moment, beans offers either no valuation or valuation at market prices. Valuation at cost is yet to be implemented.
3.4 Transaction-level valuation
beans applies valuation at the level of transactions, which seems unique amongst plain-text accounting tools. In the examples above, if a conversion to a common commodity is desired, beans will:
- convert all bookings over the entire ledger to that commodity at market value (given by the transitive & reflexive closure of price directives at the transaction date),
- insert valuation bookings for the accumulated positions of all asset and liability accounts whenever prices change. These adjustment are balanced against an Equity:Valuation account, which is hard-coded for the moment.
Every difference in value of an account over a period is explicitly modeled by corresponding valuation transactions. This greatly facilitates introspection and analysis, as every unit of the valuation commodity can be accounted for explicitly.
4 Developing beans
Development is very much in progress and beans is not really stable yet. It seems though that the basic pipeline (importing - categorizing - balancing) is there, and I am using it to balance my own journal. beans development is at the moment entirely driven by my own needs. It has also been a playground for experiments and lots of refactoring for fun with Haskell.
Use beans at your own risk - at the moment, it is certainly advantageous if you know some Haskell.
4.1 Roadmap
beans still has long way to go. I do this in my spare time and I do
not have an explicit road map, but here are some ideas I'd like to be
working on in the future:
4.1.1 Shorter term
[X]new command to print and filter transactions (as-is or at market value)[ ]provide a more realistic example file[X]multi-period reports (yearly, monthly, weekly, daily, …)[ ]colored output[ ]friendlier command line[ ]more importers (I'll do those that I need personally, but I am happy to receive contributions if they are provided with a sample file / test)[X]export to other tools (beancount /hledger/ledger)
4.1.2 Longer term
[ ]general reporting infrastructure[ ]more reports (income statement, cash flow statement)[ ]plotting / graphs[ ]portfolio performance calculation[ ]more UI options (web, terminal)
4.2 Documentation
4.3 Dev setup
4.3.1 nix / Haskell IDE engine
4.3.2 Stack
It should be possible to develop beans with stack. Please file a bug
report / pull request if you discover anything that does not work as
expected.
4.4 Adding an importer
beans is open to contributions, see the example here how an importer
can be implemented. The directory should match the country of the
bank. Make a pull request if you want to have it included in beans
(preferrably with a test case and an anonymized sample file).
It is still an open question whether importers should be included in
the beans repository or whether a more modular system should be
adopted. I am open to suggestion.
4.5 Tests
Every importer should have a test and an example file. This being said, test coverage is currently lacking for the existing importers.
5 License
beans is licensed under the BSD-3 license.