Aprendizado da máquina de troca de opções

Negociação de opções algorítmicas 1.
Apesar dos muitos recursos interessantes das opções, os comerciantes privados raramente se aproveitam (claro que eu estou falando aqui de opções sérias, e não de opções binárias). Talvez as opções sejam impopulares devido à sua reputação de ser complexas. Ou devido à sua falta de suporte pela maioria das ferramentas de software de negociação. Ou devido às etiquetas de preço das poucas ferramentas que os suportam e dos dados históricos que você precisa para negociação algorítmica. Qualquer que seja o # 8211; recentemente fizemos vários contratos de programação para sistemas de negociação de opções, e fiquei surpreso que mesmo sistemas simples pareciam produzir lucros relativamente consistentes. Especialmente as opções de venda aparecem mais lucrativas do que a negociação / convencional & # 8217; instrumentos. Este artigo é o primeiro de uma mini-série sobre ganhar dinheiro com negociação de opções algorítmicas.
Opções 101.
As opções são explicadas em muitos sites e em muitos livros de negociação, então aqui é apenas uma visão geral rápida. Uma opção é um contrato que dá ao seu proprietário o direito de comprar (opção de compra) ou vender (opção de venda) um ativo financeiro (o subjacente) a um preço fixo (o preço de exercício) em ou antes de uma data fixa (data de caducidade) . Se você vende uma opção curta (escreva), você está tendo o outro lado do comércio. Então, você pode entrar em uma posição de 4 maneiras diferentes: comprar uma ligação, comprar uma venda, vender uma chamada curta, vender uma curta. E isso com todas as combinações possíveis de preços de exercício e datas de caducidade.
O prémio é o preço que você paga ou coleciona para comprar ou vender uma opção. É muito inferior ao preço do estoque subjacente. Os principais mercados de opções geralmente são liquidos, então você pode comprar, escrever ou vender qualquer momento com qualquer preço de exercício razoável e data de validade. Se o preço subjacente atual (o preço à vista) de uma opção de compra estiver acima do preço de exercício, a opção está no dinheiro; caso contrário, está fora do dinheiro. O contrário é verdadeiro para colocar opções. In-the-money é bom para o comprador e ruim para o vendedor. As opções no dinheiro podem ser exercidas e são então trocadas pelo subjacente ao preço de exercício. A diferença de local e greve é ​​o lucro do comprador e a perda do vendedor. As opções de estilo americano podem ser exercidas a qualquer momento, opções de estilo europeu apenas no vencimento.
As opções fora do dinheiro não podem ser exercidas, pelo menos não com lucro. Mas eles não são inúteis, já que eles ainda têm a chance de entrar no dinheiro antes do vencimento. O valor de uma opção depende dessa chance e pode ser calculado para opções européias de preço à vista, greve, caducidade, taxa de rendimento sem risco, taxa de dividendos e volatilidade subjacente com a famosa fórmula de Black-Scholes. Esse valor é a base da opção premium. O verdadeiro prémio pode desviar ligeiramente devido à oferta, demanda e tentativas de prever a tendência de preços subjacentes.
Ao reverter a fórmula com um processo de aproximação, a volatilidade pode ser calculada a partir do prémio real. Esta volatilidade implícita é como o mercado espera que o subjacente flutue na próxima vez. As derivadas parciais do valor da opção são os gregos (Delta, Vega & # 8211; don & # 8217; t sabe o que a letra grega deve ser & # 8211; e Theta). Eles determinam em que direção e quão forte, o valor irá mudar quando um parâmetro de mercado muda.
Aquela & # 8217; s todas as informações básicas necessárias para opções de negociação. Por sinal, é interessante comparar os desempenhos das estratégias dos livros comerciais. Embora os sistemas de negociação forex ou de estoque descritos nesses livros sejam principalmente de beliche e perca já em um backtest simples, não é assim com os sistemas de opção. Eles muitas vezes ganham no backtests. E isso, embora eu tenha certeza de que quase nenhum autor realmente os testou. Os autores de livros de negociação de opções são apenas mais inteligentes do que outros autores de livros comerciais? Talvez, mas nós veremos que há uma explicação alternativa.
Por que as opções de negociação?
Eles são mais complexos e mais difíceis de negociar, e você precisa de uma fórmula vencedora do Prêmio Nobel para calcular um valor que de outra forma seria simplesmente uma diferença de preço de entrada e saída. Apesar de tudo isso, as opções oferecem muitas vantagens maravilhosas em relação a outros instrumentos financeiros:
Alta alavancagem. Com US $ 100 você pode comprar apenas algumas ações, mas opções de várias centenas de ações. Risco controlado. Uma posição curta em um estoque pode limpar sua conta; As posições nas opções podem ser inteligentes combinadas para limitar o risco de qualquer maneira desejada. E, ao contrário de uma perda de parada, é um limite de risco real. Dimensões adicionais. Os lucros obtidos apenas dependem da subida ou queda dos preços. Os lucros das opções podem ser alcançados com o aumento da volatilidade, a queda da volatilidade, os preços se deslocam em um intervalo, fora de um intervalo ou quase qualquer outro comportamento de preços imagináveis. Fogo e esqueça. As opções expiram, então você não precisa de um algoritmo para fechá-las (a menos que você queira vender ou exercê-las em condições especiais). E você não paga nenhuma comissão de saída por uma opção expirada. Vantagem do vendedor. Devido ao prémio, as opções ainda podem produzir um lucro para o vendedor, mesmo que o subjacente se mova na direção errada.
A ética do hacker exige que você não apenas reivindique algo, mas prová-lo. Para se familiarizar com as opções, deixe colocar a última reclamação, o vendedor aproveita para testar:
Este é um sistema de troca de opções muito simples. Ele escreve aleatoriamente opções de chamada ou colocação e mantém as posições abertas até que expiram. Devido à aleatoriedade de colocar / chamar é agnóstico de tendências. Antes de pesquisar os detalhes do código, basta executá-lo no modo [Teste] algumas vezes (você precisará do Zorro versão 1.53 ou superior). Você notará que o resultado é diferente em qualquer momento, mas é mais frequentemente positivo do que negativo, mesmo que a comissão seja subtraída do lucro. Um resultado típico:
Você pode ver que a maioria dos negócios ganha, mas quando eles perdem, eles perdem grandes. Agora, inverta a estratégia e compre as opções em vez de vendê-las: Substitua enterShort () por enterLong (). Execute-o novamente algumas vezes (o script precisa de cerca de 3 segundos para um backtest). Você verá agora que o resultado é mais freqüentemente negativo. na verdade, quase sempre.
Parece que as opções, pelo menos, os contratos SPY testados, de fato, favorecem o vendedor. Isso é um pouco semelhante à expectativa positiva de posições longas em ações, ETFs ou futuros de índice, mas as vantagens do vendedor de opções são mais fortes e independentes da direção do mercado. Isso pode explicar uma grande parte dos resultados positivos dos sistemas de opções em livros de negociação. Por que há compradores de opções? As opções são muitas vezes compradas sem fins lucrativos, mas como um seguro contra tendências de preços desfavoráveis ​​do subjacente. E por que a vantagem do vendedor não é arbitrada pelos tubarões do mercado? Talvez porque não há muitas negociações algorítmicas com opções, e porque há de qualquer forma mais baleias do que tubarões nos mercados financeiros.
Funções para opções.
Podemos ver que as opções de negociação e backtesting requer algumas mais funções do que apenas negociar o subjacente. Sem opções, o mesmo sistema de comércio aleatório seria reduzido a esse breve script:
As opções exigem (pelo menos) três funções adicionais:
dataLoad (1, & # 8221; SPY_Options. t8 & # 8243 ;, 9) carrega dados de opções históricas do arquivo & # 8220; SPY_Options. t8 & # 8221; em um conjunto de dados. Os dados de opções incluem não apenas os preços de oferta e oferta, mas também o preço de exercício, a data de validade, o tipo & # 8211; colocar ou ligar, americano ou europeu de qualquer opção, e alguns dados adicionais raramente utilizados, como o interesse aberto. Ao contrário dos preços históricos, os dados das opções geralmente são caros. Você pode comprá-lo de fornecedores como iVolatility. Mas existe uma maneira alternativa de obtê-lo gratuitamente, que eu descreverei abaixo.
A coluna do centro lista os preços de exercício e as datas de validade diferentes, as partes direita e esquerda são os preços de oferta e de oferta e os tamanhos de livro de pedidos para a chamada atribuída (esquerda) e as opções de venda (à direita). Os preços são por ação; um contrato de opção sempre cobre um certo número de ações, normalmente 100. Então, você pode ver na lista acima que você coletará $ 15 premium quando você escrever uma opção de chamada SPY que expira na próxima semana (03 de fevereiro de 2017) com US $ 230 preço de exercício. Se a SPY não ganhou mais de $ 230 até essa data, os US $ 15 são seus lucros. Se ele resultou em US $ 230 e 10 centavos e a opção é exercida (acontece automaticamente quando expira no dinheiro), você ainda mantém US $ 5. Mas se de repente subisse para US $ 300 (talvez Trump anunciasse novos muros ao redor dos EUA, tudo pago por si mesmo), você deve suportar uma perda de $ 6985.
A imagem exibe 54 contratos, mas esta é apenas uma pequena parte da cadeia de opções, pois há muitas datas de caducidade e preços de exibição disponíveis. A cadeia de opções SPY pode conter até 10.000 opções diferentes. Todos eles são baixados para o PC com a função contractUpdate acima, o que pode demorar alguns segundos a ser concluído.
contrato (Tipo, 30, preçoClose ()) seleciona uma opção específica da cadeia de opções baixada anteriormente. O tipo (PUT ou CALL), os dias até a expiração (30) e a greve (priceClose () é o preço atual do subjacente) são informações suficientes para selecionar a melhor opção de ajuste. Note-se que, para obter preços de exercício corretos no backtest, baixamos os dados de preços subjacentes com a bandeira UNADJUSTED. Os preços de greve são sempre desajustados.
Uma vez que um contrato é selecionado, o próximo enterLong () ou enterShort () compra ou vende a opção no mercado. A cláusula if () verifica se o contrato está disponível e a data de expiração é diferente do anterior (para garantir que somente contratos diferentes sejam negociados). Os limites de entrada, de paragem ou de lucro funcionariam como de costume, agora só se aplicam ao valor da opção, o prémio, em vez do preço subjacente. O backtest pressupõe que quando uma opção é exercida ou expira no dinheiro, o subjacente é vendido imediatamente e o lucro é registrado na conta do comprador e deduzido da conta do vendedor. Se a opção expirar para fora do dinheiro, a posição simplesmente desaparece. Portanto, não nos preocupamos com a saída de posições nesta estratégia. Além dessas diferenças, as opções de negociação funcionam de acordo com a negociação de qualquer outro instrumento financeiro.
Estratégias de opções de backtesting.
Aqui é uma maneira fácil de se enriquecer. Abra uma conta do IB e execute um software que registre as cadeias de opções e os preços do contrato em intervalos de um minuto. Isso é o que alguns fornecedores de dados fizeram nos últimos 5 anos, e agora eles estão querendo vender seus tesouros de dados. Embora você possa facilmente pagar vários milhares de dólares por algumas cadeias de opções de ações principais, não tenho certeza de quem realmente possui os direitos autorais desses dados # 8211; o vendedor, o corretor, a troca ou os participantes do mercado? Esta pode ser uma área cinzenta legal. De qualquer forma, você precisa de dados históricos para o desenvolvimento de estratégias de opções, caso contrário, você não pode fazer o backtest deles.
Aqui é um método para obtê-lo de graça e sem problemas legais:
Este script é um pouco mais longo do que os scripts Zorro habituais que postei aqui, então eu não o expliquei detalhadamente. Ele gera cadeias de opções artificiais para qualquer dia a partir de 2018-2017 e as armazena em um arquivo de dados histórico. Os preços das opções são calculados a partir do preço subjacente, da volatilidade, da taxa de juros livre de risco atual e da taxa de dividendos do subjacente. Ele usa três faixas de preços de exercício e datas de caducidade em qualquer sexta-feira dos próximos 180 dias. Você precisa de R instalado para executá-lo, e também o pacote RQuantlib para calcular os valores das opções. Todas as funções são descritas no manual Zorro. A função yield () retorna a taxa de rendimento atual das contas do Tesouro dos EUA e contractVal () calcula o prémio ao resolver uma equação diferencial com todos os parâmetros da opção. O código-fonte de ambas as funções pode ser encontrado no arquivo de contrato. c incluir.
Devido ao solucionador de equações diferenciais lentas e ao grande número de opções, o script precisa de várias horas para ser concluído. Aqui é uma comparação dos dados gerados com dados reais de opções SPY:
A linha azul são os preços das opções artificiais, a linha preta são os preços reais comprados de um fornecedor de dados de opções, tanto para contratos SPY de 3 semanas com 10 pontos de distância spot-strike. Você pode ver que os preços combinam bastante bem. Existem pequenas diferenças que podem ser parcialmente aleatórias, parcialmente causadas por anomalias na oferta e na demanda. Para estratégias que exploram essas anomalias & # 8211; que inclui todas as estratégias baseadas em volatilidade implícita e # 8211; Você precisará de preços reais de opções históricas. Para as estratégias de opções que exploram apenas as mudanças de preço ou de volatilidade do subjacente, os dados artificiais provavelmente irão fazer. Veja, lendo este artigo até o final, você já salvou alguns milhares de dólares.
Conclusão.
Opções e combinações de opções podem ser usadas para criar instrumentos financeiros artificiais com propriedades muito interessantes. As estratégias de opções, especialmente as opções de venda, são mais propensas a serem lucrativas do que outras estratégias. As estratégias de opções algorítmicas são um pouco, mas não muito mais complexas do que as estratégias com outros instrumentos financeiros.
Eu incluí todos os scripts no repositório de script de 2017 e também um conjunto de dados históricos com as taxas de rendimento (caso contrário, você precisava da ponte Quandl ou do Zorro S para baixá-los). Você precisará do Zorro 1.53 ou superior, atualmente disponível no & # 8220; Beta & # 8221; link da página de download do Zorro. A mensagem de erro da versão Zorro gratuita sobre a ponte Quandl não suportada pode ser ignorada, devido às taxas de rendimento incluídas, o script será executado no entanto.
No próximo artigo, analisaremos mais de perto os valores das opções e os métodos para combinar opções para limitar o risco ou negociar intervalos de preços arbitrários. Essas combinações com nomes engraçados como "Iron Condor" e # 8221; ou & # 8220; Borboleta & # 8221; são muitas vezes referidos como estratégias de opções, mas não são & # 8211; são apenas instrumentos financeiros artificiais. Como você os troca é até a estratégia real. Algumas estratégias de opções simples, mas consistentemente rentáveis ​​serão o tema do terceiro artigo desta mini-série.
49 pensamentos sobre & ldquo; Algorithmic Options Trading 1 & rdquo;
Artigo muito interessante Eu tenho um sistema de troca automático de opções criado pelos desenvolvedores do Zorro (ótimo trabalho, por sinal) e é muito interessante ver que minha estratégia gera resultados semelhantes à sua estratégia # 8220; aleatória # 8221 ;. Estou ansioso para os próximos artigos desta mini-série.
Gostaria de perguntar, você tem alguma idéia se seu livro será traduzido para o inglês em breve? Adoraria ler o livro.
Eu estou totalmente interessado nestes mini artigos da série. Por favor, deixe-me saber a próxima série.
Obrigado # 8211; sim, uma versão de livro em inglês está planejada, eu só devo encontrar algum tempo para revisar a tradução bruta. Andrés: você pode inserir seu e-mail no campo de inscrição à direita.
Bom artigo, gostaria de lhe perguntar o que são bons livros ou onde posso aprender a negociar com opções. Obrigado.
Estou certo, porque esses preços artificiais e reais se relacionam com uma espécie de sintético & # 8221; opção feita como uma série rolada de opções reais com a data de validade mais próxima e greve dinamicamente alterada (dependendo do preço subjacente)?
Investopedia e Tastytrade têm alguns tutoriais e vídeos sobre opções. - Não foi lançada a série, mas uma cadeia de opções com diferentes greves e datas de expiração, assim como na vida real. Caso contrário, o backtest não seria realista.
Quando você está comparando os preços artificiais com os preços reais, você está usando ataque ATM? O ponto inteiro, para mim, de testar uma estratégia de negociação de opções versus dados de opções reais é que, nas asas, os volumes implícitos serão muito superiores aos gerados artificialmente.
As greves utilizadas foram cerca de 10 pontos ITM.
Obrigado por publicar este interessante artigo. Posso saber quando os outros dois artigos desta mini-série serão publicados?
Quando eu tiver algum tempo & # 8230; 🙂
Que bom artigo! Os resultados do sistema de comércio aleatório são semelhantes aos CBOE S & amp; P 500 PutWrite Index e faz sentido.
Muito obrigado por este artigo! Estava pensando nisso no outro dia.
Eu gosto muito dos artigos deste blog. Atualmente, estou negociando opções de compra de prazo de 1 ano de ações específicas.
Meu maior problema com a vantagem do vendedor & # 8221; que contradiz o risco controlado & # 8221; declaração.
& # 8220; Algo que muitas vezes confunde os investidores é se, ou não, ser uma chamada curta e uma longa colocação são iguais. Intuitivamente, isso pode ter algum sentido, uma vez que as chamadas e colocações são contratos quase opostos, mas ser uma chamada curta e um longo tempo não é o mesmo. Quando você é comprido, você tem que pagar o prêmio e o pior caso resultará em perda do prêmio. No entanto, quando você recebe uma chamada curta, você coleciona a opção premium, mas você está exposto a uma grande quantidade de risco & # 8221;
Então, quando você escreve (nua), seu risco é ilimitado. O curto período de tempo de expiração (30 dias) é salva-lo na maioria dos casos, mas isso é uma auto-ilusão. Este método é muito semelhante aos bots de negociação de fraude, onde 99,5% dos bots do tempo estão ganhando pouco (e. G. Call premium) quantidade de dinheiro, no entanto, quando você perde, você arrisca grande quantidade de seu dinheiro.
O risco prolongado ou o risco de comerciantes são limitados e eles escolhem opções fora do dinheiro para multiplicar seus ganhos e, paralelamente, eles reduzem sua chance vencedora.
Eu estaria interessado em LEAPS (1+ ano de expiração longo / put opções) backtest.
Apenas faça isso. Faça o download do Zorro 1.54 no fórum do usuário e execute um sistema com o LEAPS. Para isso, você precisa aumentar o & # 8220; DaysMax & # 8221; variável no script de geração de dados de opções acima de 1 ano (365) ou 2 anos (2 * 365) para incluir contratos de longo prazo. O script precisará um pouco mais de tempo para a geração de dados.
Uma vez que as opções de negociação são um novo recurso Zorro, eu estou me perguntando se a parte do manual Broker do manual (zorro-trader / manual / en / brokerplugin. htm) foi suficientemente atualizada para atender as opções de manipulação.
Eu estou pedindo porque eu estou tentando escrever um plugin DLL para TradeKing (em breve para ser renomeado para Ally Invest). Eles possuem ações, ETFs e contratos de opções. Corretor muito baixo de barreira para entrada também ($ 0 necessário para obter acesso à API).
Para opções, implemente as funções básicas da API mais 5 funções BrokerCommand: GET_POSITION, GET_OPTIONS, GET_UNDERLYING, SET_SYMBOL e SET_MULTIPLIER.
Artigo fantástico, obrigado por compartilhar, testei o código e baixei os dados das opções através do script, tudo pareceu fazer o download de OK e me fazer um arquivo T8 de 48mb para o SPY, mas quando eu executar o script aleatório, não obtenho quaisquer negociações. É a primeira vez que eu corri o zorro (I & # 8217; m na última versão baixada há 2-3 dias), por isso realmente não tenho certeza do que eu estou fazendo de errado.
Qualquer ajuda será apreciada e espero ansiosamente o próximo episódio nesta série fascinante 😉
Aqui está a saída do log:
Opções de testeSellRandom SPY.
Conta simulada AtivosIB.
Período de barra 24 horas (média 2233 min)
Período de teste 12.01.2018-01.06.2018 (1270 bars)
Período de busca 80 bares (16 semanas)
Modo de simulação realista (deslizamento 5,0 segundos)
Spread 2.0 pips (roll 0.00 / 0.00)
Contratos por lote 1.0.
Perda / perda bruta 0,00 $ / -0,00 $ (-1p)
Lucro médio de 0,00 $ / ano, 0,00 $ / mês, 0,00 $ / dia.
Dispensa máxima -0.00 $ -1% (MAE -0.00 $ -1%)
Tempo de inatividade total 0% (TAE 0%)
Tempo máximo de queda 0 minutos a partir de setembro de 2018.
Margem máxima aberta 0.00 $
Risco máximo aberto 0,00 $
Volume comercial 0,00 $ (0,00 $ / ano)
Custos de transação 0.00 $ spr, 0.00 $ slp, 0.00 $ rol.
Capital requerido 0 $
Número de negócios 279 (52 / ano, 1 / semana, 1 / dia)
Percentagem de ganhos de 0,0%
Vitória / perda máxima 0.00 $ / & # 8211; 0.00 $
Lucro médio de lucro 0,00 $ -1. $ P (+ 0.0p / -1. $ P)
Deslizamento do comércio médio 0,00 $ 1. $ p (+ 0.0p / -1. $ P)
Barras de comércio médio 23 (+0 / -23)
Barras comerciais máximas 26 (5 semanas)
Tempo no mercado 506%
Negociações abertas máximas 6.
Raio de perda máxima 279 (não correlacionado 279)
Retorno anual 0%
Taxa Sharpe 0,00.
Critério de Kelly 0,00.
R2 coeficiente 1.000.
Nível de confiança AR DDMax Capital.
Análise de portfólio OptF ProF Win / Loss Wgt%
e um trecho do arquivo de log & # 8230;
[1338: Sex 13.05.16 19:00] +0 +0 6/271 (206.21)
[SPY :: SC1272] Ligue para 20180513 204.0 0@3.5713 não negociado hoje!
[SPY :: SC1272] Expirou 1 Ligue 20180513 204.0 0 @ 207: +0.00 às 19:00:00.
[1339: Seg 16.05.16 19:00] +0 +0 5/272 (204.96)
[1340: Ter 17.05.16 19:00] +0 +0 5/272 (206.46)
[1341: Qua 18.05.16 19:00] +0 +0 5/272 (204.44)
[1342: Qui 19.05.16 19:00] +0 +0 5/272 (204.06)
[SPY :: SC4278] Escreva 1 Ligue 20180624 205.0 0@3.4913 às 19:00:00.
[1343: Sex 20.05.16 19:00] +0 +0 6/272 (204.92)
[SPY :: SP1773] Coloque 20180520 208.0 0@4.2851 não negociado hoje!
[SPY :: SP1773] Expirou 1 Coloque 20180520 208.0 0 @ 204: +0.00 às 19:00:00.
[1344: Seg 23.05.16 19:00] +0 +0 5/273 (205.51)
[1345: Ter 24.05.16 19:00] +0 +0 5/273 (206.17)
[1346: Qua 25.05.16 19:00] +0 +0 5/273 (208.67)
[1347: Qui 26.05.16 19:00] +0 +0 5/273 (209.44)
[SPY :: SC4779] Escreva 1 Ligue 20180701 209.0 0@3.7358 às 19:00:00.
[1348: Sex 27.05.16 19:00] +0 +0 6/273 (209.53)
[SPY :: SP2274] Coloque 20180527 208.0 0@3.3622 não negociado hoje!
[SPY :: SP2274] Expirou 1 Coloque 20180527 208.0 0 @ 209: +0.00 às 19:00:00.
[1349: Ter 31.05.16 19:00] +0 +0 5/274 (210.56)
[SPY :: SC2775] Capa 1 Ligue para 20180531 207.0 0@2.2309: +0.00 às 19:00:00.
[SPY :: SC3276] Capa 1 Ligue 20180531 205.0 0@5.1843: +0.00 às 19:00:00.
[SPY :: SP3777] Capa 1 Coloque 20180531 206.0 0@0.8602: +0.00 às 19:00:00.
[SPY :: SC4278] Capa 1 Ligue para 20180531 205.0 0@4.9463: +0.00 às 19:00:00.
[SPY :: SC4779] Capa 1 Ligue 20180531 209.0 0@2.8347: +0.00 às 19:00:00.
[1350: Qua 01.06.16 19:00] +0 +0 0/279 (209.12)
Eu vejo que as posições são todas abertas com zero volume, como se você tivesse definido a quantidade de contratos para 0. Você usou o script não modificado do repositório?
I & # 8217; m usando o arquivo OptionsSimulate. c diretamente do arquivo Zip.
Eu instalei R e as bibliotecas Quantlib e a ponte R pareceu funcionar bem também.
O topo do arquivo.
string FileName = & # 8220; Histórico \\ SPY_SimOptions. t8 & # 8221 ;;
var StrikeMax [3] =; // 3 intervalos de ataque com diferentes passos.
var StrikeStep [3] =; // larguras de passo para os 3 intervalos.
int DaysMax = 180;
var BidAskSpread = 2.5; // Bid / Ask spread em percentagem.
var Dividendo = 0,02;
int Type = 0; // ou EUROPEO, ou FUTURO.
LookBack = 21; // por volatilidade.
Lamentamos as perguntas do n00b, são ferramentas e sistemas realmente interessantes e queria testar alguns spreads de crédito verticais usando este código como base para o SPY e talvez alguns outros instrumentos!
Não é uma questão de noob, na verdade é minha culpa. Eu apenas vejo que eu esqueci de definir as opções multiplicadoras no script. Isso não importava com a versão anterior do Zorro, uma vez que o multiplicador era 100 por padrão, mas agora deve ser configurado porque as opções podem ter multiplicadores muito diferentes.
I & # 8217; corrigiu o script acima. Obrigado por me notificar!
Sim, foi isso!
Obtendo resultados agora, muito obrigado pela sua ajuda jcl.
I & # 8217; m agora para colocar $ 1mm em uma conta e trocar este bebê 😉
Você tem alguma idéia quando você vai começar a trabalhar no resto dos artigos desta série?
Parece que o código abaixo não está funcionando mais.
O arquivo CSV SPY. csv é preenchido com este conteúdo:
QECx05, O URL que você solicitou está incorreto. Use a seguinte URL em vez disso: / api / v3 / datasets /: database_code /: dataset_code.
Desculpe, na verdade, esse arquivo era de Quandl, e precisa de uma assinatura paga.
Do Yahoo eu recebo o erro Can & # 8217; t baixar o SPY do Yahoo.
Alguém que tenha o mesmo problema?
Eu acho que todos estão tendo o mesmo problema, já que o Yahoo mudou seu protocolo na semana passada. Se você se deparar com problemas como esse, procure uma solução não só no meu blog, mas primeiro no fórum Zorro:
Obrigado por esta informação útil sobre sistemas de negociação automatizados!
Eu sou muito novo para isso, mas acho que este é um negócio muito maior do que você faz parecer som:
& gt; Existem pequenas diferenças que podem ser parcialmente aleatórias, parcialmente causadas por anomalias na oferta e na demanda. Para estratégias que exploram essas anomalias, você precisará de dados históricos reais.
Ter uma volatilidade precisa é essencial. Sem isso, você não está apenas escrevendo uma estratégia que não explore essas anomalias, você está escrevendo uma que as ignora completamente. É comparável ao gerar o preço de estoque, escolhendo um número aleatório com base na distribuição de probabilidade das semanas anteriores e # 8217; preços ou alisando todos os maiores movimentos.
Os preços das opções são baseados em expectativas sobre o futuro, mas (a menos que eu não entenda seu código), você está avaliando-os com base no passado. As diferenças serão mais pronunciadas em subjacentes diferentes de SPY, particularmente em torno do tempo de renda (digamos AAPL, MSFT ou GOOG).
Também acho difícil pensar em uma estratégia que não explore a diferença entre a volatilidade implícita e real. Mesmo um delta de 16/5 colocado na SPY só funciona tão bem como porque a IV é muito mais alta do que deveria ser.
Sim, as mudanças nos preços das opções devido à expectativa de volatilidade, talvez quando a abordagem da notícia da empresa pertence às anomalias mencionadas. A regra geral é: para anomalias que também têm um efeito sobre o subjacente, você pode usar os preços artificiais. Para anomalias que afetam apenas as opções, mas não o subjacente, você precisará comprar dados das opções históricas reais.
Quão bons serão os dados simulados se eu mudar o BarPeriod = 1440 para ser BarPeriod = 1?
Teoricamente, tão bom ou ruim quanto os dados diários, já que o priciple é o mesmo. Mas eu ainda não fiz testes com dados de opções de 1 minuto. Isso é uma grande quantidade de dados.
& # 8220; Devido ao solucionador de equação diferencial lenta e ao grande número de opções, o script precisa de várias horas para concluir. & # 8221;
Quanto mais rápido você acha que isso poderia ser se o R / Quantmod fosse substituído por C / C ++? Estou pensando em gerar muitos dados sintéticos.
Eu acredito que ele é _ C ++, pelo menos o Quantlib subjacente está programado em C ++. A sobrecarga R é provavelmente insignificante. O problema não é o código, mas a matemática. A resolução numérica de equações diferenciais é lenta. Black-Scholes é muito mais rápido, mas apenas para opções europeias. Se você realmente possui muitos dados para gerar, pode fazer sentido verificar a velocidade de diferentes métodos de aproximação para opções americanas.
Percebo que a volatilidade é fixada em 20 no script acima para gerar preços de opções sintéticas. Poderia não haver um argumento para que a volatilidade seja 30 dias e calculada programaticamente a partir do subjacente?
O que você quer dizer com & # 8220: 30 dias e # 8221 ?? 20 é o período de volatilidade usual nos cálculos financeiros, já que equivale aproximadamente a um mês. 30 provavelmente não faria muita diferença.
Você usa uma estimativa única de Volatilidade, eu acho: por exemplo, 16 para o S & amp; P. Mas, de forma contínua, será muito amplamente, o que é, naturalmente, parte do motivo pelo qual os preços das opções mudam tanto: como a volatilidade aumenta, o preço da opção também. Se, portanto, você usar uma média móvel de volatilidade de 20 (ou 30) dias, você obterá preços de opções sintéticas mais precisos do que simplesmente assumir um plano 16 único para o S & amp; P quando às vezes o real pode ser 10, às vezes 30. Não tenho olhou para a arquitetura do zorro e, portanto, não é agora se é principalmente vetor, ou olhar ou o que. De qualquer forma, seria possível incluir a média móvel do dia relevante da volatilidade do instrumento subjacente ao invés de uma figura fixa.
Mas lá novamente é o que você faz, talvez? HistVolOV = VolatilidadeOV (20) & # 8211; talvez este seja 20 dias? Não é 20%?
Uma pergunta não é uma declaração.
De qualquer forma, parece um maravilhoso software. Apenas vou seguir o manual.
Sim, parece Vol é uma série temporal. Desculpe incomodá-lo.
Sim, a volatilidade anualizada dos últimos 20 dias. Se fosse 20%, eu teria escrito: HistVolOV = 0,2.
Não. Não o corta. Você não pode usar uma única medida de volatilidade histórica para tudo, desde uma opção de um mês até um prazo de validade de 24 meses. Talvez o esquema inteiro seja inválido. Por exemplo, IV para um vencimento de dois anos do SPX é atualmente de 15%, enquanto uma opção que expira nos próximos dias é de 5% de ish.
Pode ser inválido usar dados manufaturados. Exceto se você tratá-lo como uma espécie de teste de Monte Carlo: isso é o que pode / poderia ter acontecido / pode acontecer.
Anthony, o script está calculando o preço atual de uma opção. O preço atual depende da volatilidade atual. Não há volatilidade de 24 meses atrás.
Você calcula o valor das opções européias com a fórmula Black Scholes e as opções americanas, como no script acima, com um método de aproximação. Ambos os métodos normalmente usam 20 dias de volatilidade. O método de amostragem de volatilidade pode ser diferente, mas os 20 dias são bastante comuns para todas as opções de software de negociação que eu conheço. E você pode ver a partir da comparação com preços reais acima que esse período funciona bastante bem.
Não, você não pode calcular o preço atual de uma opção em um determinado dia dessa maneira. Não há como reproduzir com precisão a volatilidade implícita, portanto, o preço em qualquer data no passado. E é a volatilidade implícita em que nos interessa, e não o histórico. Eu concordo totalmente sobre Black Scholes, é claro, e os seus usos, mas é carrinho antes do cavalo esperar para ligar a volatilidade de 20 dias em 3 de janeiro de 1985 e esperar que ele venha um preço exato negociado no fechamento naquele dia para o SPX para qualquer greve ou expiração.
Está olhando para ele no caminho errado.
O que você pode tentar é brincar com diferentes métodos para estimar o que o implícito vol / preço pode ter ocorrido em 3 de janeiro de 1985 para uma determinada greve e expiração de uma opção SPX.
Por exemplo, você pode usar uma volatilidade histórica de 5 dias para uma opção que expira em uma semana e uma volatilidade de 252 dias para uma opção que expira em um ano. Ou você pode implicar volatilidades ao analisar a estrutura de prazo dos contratos de futuros da VIX a partir de 2004. Ou, pelo menos, usar o próprio índice VIX, que retorna a 1986 como entrada para a volatilidade de 30 dias.
Seja lá o que fizer, você ganhou realmente estar produzindo algo como o que realmente foi negociado no dia. Ou, pelo menos, não de forma consistente e precisa em todos os períodos e greves.
Eu acredito que o processo que você descreve tem um valor, mas que o resultado de ambos os preços produzidos e os testes de retorno resultantes disso serão mais parecidos com um processo aleatório de moet carlo do que com um teste de volta nos dados de preços negociados reais.
Eu acredito que é um processo valioso, mas que o que é produzido é uma série de universos paralelos: o que poderia ter acontecido com uma determinada estratégia ao longo de um determinado período de tempo usando volatilidades implícitas que podem ou não ter sido negociadas.
Desculpe-me por muito tempo e sou um admirador do seu produto e do seu script acima. Eu não teria pensado em gerar preços falsos de opções se não tivesse visto o seu excelente artigo.
Mas, na minha opinião, pelo menos você precisa repensar sua contribuição para a fórmula BS quanto à volatilidade.
Aliás, fique bem ciente de que admiro seu produto e seus pensamentos. Não imagino que estou sendo difícil. Igualmente, por favor, não imagine, acredito que eu seja # 8220; certo e # 8221 ;!
Estou apenas aproveitando a jornada e o diálogo com você e espero que juntos possamos melhorar a compreensão do tema.
O meu é limitado!
Diga a data em que você está procurando a 7 de janeiro de 1987. Naquele dia, a volatilidade histórica de SPX calculada em mais de 20 dias de negociação foi de 15,23. A volatilidade histórica nesse dia nos últimos 252 dias foi de 14,65.
For 5 days it was 18.
Now say I am trying to “calculate” (guess) a price (which might have been traded on 7th January 1987) for an option expiring in 5 days, 20 days and 252 days. Lets assume ATM.
My suspicion is that it would not be helpful to use 15.23 for all three expiries.
Thank you for your kind words. Finance is complex. My knowledge is even more limited and I’m daily surprised by some results that I didn’t expect. & # 8211; In your example, the 15.23% volatility is the correct value. If you used a higher volatility period for higher expiration, then it depends on whether it’s still annualized volatility or just volatility of a longer time. In the latter case the results are off by some factor, in the former case they are based on too old volatility and thus not up to date. & # 8211; You’re right about the implied volatility, since it is affected by the difference of theoretical and real option value. So you cannot use the script above for getting it. Otherwise you would just get back some approximation of the current volatility. You need real option prices for IV.
I hope that it’s alright that I discuss this with just a few of my clientele, this will assist.

Melhores estratégias 5: um sistema de aprendizado de máquina a curto prazo.
It’s time for the 5th and final part of the Build Better Strategies series. In part 3 we’ve discussed the development process of a model-based system, and consequently we’ll conclude the series with developing a data-mining system. The principles of data mining and machine learning have been the topic of part 4. For our short-term trading example we’ll use a deep learning algorithm , a stacked autoencoder, but it will work in the same way with many other machine learning algorithms. With today’s software tools, only about 20 lines of code are needed for a machine learning strategy. I’ll try to explain all steps in detail.
Our example will be a research project – a machine learning experiment for answering two questions. Does a more complex algorithm – such as, more neurons and deeper learning – produce a better prediction? And are short-term price moves predictable by short-term price history? The last question came up due to my scepticism about price action trading in the previous part of this series. I got several emails asking about the “trading system generators” or similar price action tools that are praised on some websites. There is no hard evidence that such tools ever produced any profit (except for their vendors) – but does this mean that they all are garbage? We’ll see.
Our experiment is simple: We collect information from the last candles of a price curve, feed it in a deep learning neural net, and use it to predict the next candles. My hypothesis is that a few candles don’t contain any useful predictive information. Of course, a nonpredictive outcome of the experiment won’t mean that I’m right, since I could have used wrong parameters or prepared the data badly. But a predictive outcome would be a hint that I’m wrong and price action trading can indeed be profitable.
Machine learning strategy development.
Step 1: The target variable.
To recap the previous part: a supervised learning algorithm is trained with a set of features in order to predict a target variable . So the first thing to determine is what this target variable shall be. A popular target, used in most papers, is the sign of the price return at the next bar. Better suited for prediction, since less susceptible to randomness, is the price difference to a more distant prediction horizon , like 3 bars from now, or same day next week. Like almost anything in trading systems, the prediction horizon is a compromise between the effects of randomness (less bars are worse) and predictability (less bars are better).
Sometimes you’re not interested in directly predicting price, but in predicting some other parameter – such as the current leg of a Zigzag indicator – that could otherwise only be determined in hindsight. Or you want to know if a certain market inefficiency will be present in the next time, especially when you’re using machine learning not directly for trading, but for filtering trades in a model-based system. Or you want to predict something entirely different, for instance the probability of a market crash tomorrow. All this is often easier to predict than the popular tomorrow’s return.
In our price action experiment we’ll use the return of a short-term price action trade as target variable. Once the target is determined, next step is selecting the features.
Step 2: The features.
A price curve is the worst case for any machine learning algorithm. Not only does it carry little signal and mostly noise , it is also nonstationary and the signal/noise ratio changes all the time. The exact ratio of signal and noise depends on what is meant with “signal”, but it is normally too low for any known machine learning algorithm to produce anything useful. So we must derive features from the price curve that contain more signal and less noise. Signal, in that context, is any information that can be used to predict the target, whatever it is. All the rest is noise.
Thus, selecting the features is critical for success – much more critical than deciding which machine learning algorithm you’re going to use. There are two approaches for selecting features. The first and most common is extracting as much information from the price curve as possible. Since you do not know where the information is hidden, you just generate a wild collection of indicators with a wide range of parameters, and hope that at least a few of them will contain the information that the algorithm needs. This is the approach that you normally find in the literature. The problem of this method: Any machine learning algorithm is easily confused by nonpredictive predictors. So it won’t do to just throw 150 indicators at it. You need some preselection algorithm that determines which of them carry useful information and which can be omitted. Without reducing the features this way to maybe eight or ten, even the deepest learning algorithm won’t produce anything useful.
The other approach, normally for experiments and research, is using only limited information from the price curve. This is the case here: Since we want to examine price action trading, we only use the last few prices as inputs, and must discard all the rest of the curve. This has the advantage that we don’t need any preselection algorithm since the number of features is limited anyway. Here are the two simple predictor functions that we use in our experiment (in C):
The two functions are supposed to carry the necessary information for price action: per-bar movement and volatility. The change function is the difference of the current price to the price of n bars before, divided by the current price. The range function is the total high-low distance of the last n candles, also in divided by the current price. And the scale function centers and compresses the values to the +/-100 range, so we divide them by 100 for getting them normalized to +/-1 . We remember that normalizing is needed for machine learning algorithms.
Step 3: Preselecting/preprocessing predictors.
When you have selected a large number of indicators or other signals as features for your algorithm, you must determine which of them is useful and which not. There are many methods for reducing the number of features, for instance:
Determine the correlations between the signals. Remove those with a strong correlation to other signals, since they do not contribute to the information. Compare the information content of signals directly, with algorithms like information entropy or decision trees. Determine the information content indirectly by comparing the signals with randomized signals; there are some software libraries for this, such as the R Boruta package. Use an algorithm like Principal Components Analysis (PCA) for generating a new signal set with reduced dimensionality. Use genetic optimization for determining the most important signals just by the most profitable results from the prediction process. Great for curve fitting if you want to publish impressive results in a research paper.
For our experiment we do not need to preselect or preprocess the features, but you can find useful information about this in articles (1), (2), and (3) listed at the end of the page.
Step 4: Select the machine learning algorithm.
R offers many different ML packages, and any of them offers many different algorithms with many different parameters. Even if you already decided about the method – here, deep learning – you have still the choice among different approaches and different R packages. Most are quite new, and you can find not many empirical information that helps your decision. You have to try them all and gain experience with different methods. For our experiment we’ve choosen the Deepnet package, which is probably the simplest and easiest to use deep learning library. This keeps our code short. We’re using its Stacked Autoencoder ( SAE ) algorithm for pre-training the network. Deepnet also offers a Restricted Boltzmann Machine ( RBM ) for pre-training, but I could not get good results from it. There are other and more complex deep learning packages for R, so you can spend a lot of time checking out all of them.
How pre-training works is easily explained, but why it works is a different matter. As to my knowledge, no one has yet come up with a solid mathematical proof that it works at all. Anyway, imagine a large neural net with many hidden layers:
Training the net means setting up the connection weights between the neurons. The usual method is error backpropagation. But it turns out that the more hidden layers you have, the worse it works. The backpropagated error terms get smaller and smaller from layer to layer, causing the first layers of the net to learn almost nothing. Which means that the predicted result becomes more and more dependent of the random initial state of the weights. This severely limited the complexity of layer-based neural nets and therefore the tasks that they can solve. At least until 10 years ago.
In 2006 scientists in Toronto first published the idea to pre-train the weights with an unsupervised learning algorithm, a restricted Boltzmann machine. This turned out a revolutionary concept. It boosted the development of artificial intelligence and allowed all sorts of new applications from Go-playing machines to self-driving cars. In the case of a stacked autoencoder, it works this way:
Select the hidden layer to train; begin with the first hidden layer. Connect its outputs to a temporary output layer that has the same structure as the network’s input layer. Feed the network with the training samples, but without the targets. Train it so that the first hidden layer reproduces the input signal – the features – at its outputs as exactly as possible. The rest of the network is ignored. During training, apply a ‘weight penalty term’ so that as few connection weights as possible are used for reproducing the signal. Now feed the outputs of the trained hidden layer to the inputs of the next untrained hidden layer, and repeat the training process so that the input signal is now reproduced at the outputs of the next layer. Repeat this process until all hidden layers are trained. We have now a ‘sparse network’ with very few layer connections that can reproduce the input signals. Now train the network with backpropagation for learning the target variable, using the pre-trained weights of the hidden layers as a starting point.
The hope is that the unsupervised pre-training process produces an internal noise-reduced abstraction of the input signals that can then be used for easier learning the target. And this indeed appears to work. No one really knows why, but several theories – see paper (4) below – try to explain that phenomenon.
Step 5: Generate a test data set.
We first need to produce a data set with features and targets so that we can test our prediction process and try out parameters. The features must be based on the same price data as in live trading, and for the target we must simulate a short-term trade. So it makes sense to generate the data not with R, but with our trading platform, which is anyway a lot faster. Here’s a small Zorro script for this, DeepSignals. c :
We’re generating 2 years of data with features calculated by our above defined change and range functions. Our target is the result of a trade with 3 bars life time. Trading costs are set to zero, so in this case the result is equivalent to the sign of the price difference at 3 bars in the future. The adviseLong function is described in the Zorro manual; it is a mighty function that automatically handles training and predicting and allows to use any R-based machine learning algorithm just as if it were a simple indicator.
In our code, the function uses the next trade return as target, and the price changes and ranges of the last 4 bars as features. The SIGNALS flag tells it not to train the data, but to export it to a. csv file. The BALANCED flag makes sure that we get as many positive as negative returns; this is important for most machine learning algorithms. Run the script in [Train] mode with our usual test asset EUR/USD selected. It generates a spreadsheet file named DeepSignalsEURUSD_L. csv that contains the features in the first 8 columns, and the trade return in the last column.
Step 6: Calibrate the algorithm.
Complex machine learning algorithms have many parameters to adjust. Some of them offer great opportunities to curve-fit the algorithm for publications. Still, we must calibrate parameters since the algorithm rarely works well with its default settings. For this, here’s an R script that reads the previously created data set and processes it with the deep learning algorithm ( DeepSignal. r ):
We’ve defined three functions neural. train , neural. predict , and neural. init for training, predicting, and initializing the neural net. The function names are not arbitrary, but follow the convention used by Zorro’s advise(NEURAL. ) function. It doesn’t matter now, but will matter later when we use the same R script for training and trading the deep learning strategy. A fourth function, TestOOS , is used for out-of-sample testing our setup.
The function neural. init seeds the R random generator with a fixed value (365 is my personal lucky number). Otherwise we would get a slightly different result any time, since the neural net is initialized with random weights. It also creates a global R list named “Models”. Most R variable types don’t need to be created beforehand, some do (don’t ask me why). The ‘<<-‘ operator is for accessing a global variable from within a function.
The function neural. train takes as input a model number and the data set to be trained. The model number identifies the trained model in the “ Models ” Lista. A list is not really needed for this test, but we’ll need it for more complex strategies that train more than one model. The matrix containing the features and target is passed to the function as second parameter. If the XY data is not a proper matrix, which frequently happens in R depending on how you generated it, it is converted to one. Then it is split into the features ( X ) and the target ( Y ), and finally the target is converted to 1 for a positive trade outcome and 0 for a negative outcome.
The network parameters are then set up. Some are obvious, others are free to play around with:
The network structure is given by the hidden vector: c(50,100,50) defines 3 hidden layers, the first with 50, second with 100, and third with 50 neurons. That’s the parameter that we’ll later modify for determining whether deeper is better. The activation function converts the sum of neuron input values to the neuron output; most often used are sigmoid that saturates to 0 or 1, or tanh that saturates to -1 or +1.
We use tanh here since our signals are also in the +/-1 range. The output of the network is a sigmoid function since we want a prediction in the 0..1 range. But the SAE output must be “linear” so that the Stacked Autoencoder can reproduce the analog input signals on the outputs.
The learning rate controls the step size for the gradient descent in training; a lower rate means finer steps and possibly more precise prediction, but longer training time. Momentum adds a fraction of the previous step to the current one. It prevents the gradient descent from getting stuck at a tiny local minimum or saddle point. The learning rate scale is a multiplication factor for changing the learning rate after each iteration (I am not sure for what this is good, but there may be tasks where a lower learning rate on higher epochs improves the training). An epoch is a training iteration over the entire data set. Training will stop once the number of epochs is reached. More epochs mean better prediction, but longer training. The batch size is a number of random samples – a mini batch – taken out of the data set for a single training run. Splitting the data into mini batches speeds up training since the weight gradient is then calculated from fewer samples. The higher the batch size, the better is the training, but the more time it will take. The dropout is a number of randomly selected neurons that are disabled during a mini batch. This way the net learns only with a part of its neurons. This seems a strange idea, but can effectively reduce overfitting.
All these parameters are common for neural networks. Play around with them and check their effect on the result and the training time. Properly calibrating a neural net is not trivial and might be the topic of another article. The parameters are stored in the model together with the matrix of trained connection weights. So they need not to be given again in the prediction function, neural. predict . It takes the model and a vector X of features, runs it through the layers, and returns the network output, the predicted target Y . Compared with training, prediction is pretty fast since it only needs a couple thousand multiplications. If X was a row vector, it is transposed and this way converted to a column vector, otherwise the nn. predict function won’t accept it.
Use RStudio or some similar environment for conveniently working with R. Edit the path to the. csv data in the file above, source it, install the required R packages (deepnet, e1071, and caret), then call the TestOOS function from the command line. If everything works, it should print something like that:
TestOOS reads first our data set from Zorro’s Data folder. It splits the data in 80% for training ( XY. tr ) and 20% for out-of-sample testing ( XY. ts ). The training set is trained and the result stored in the Models list at index 1. The test set is further split in features ( X ) and targets ( Y ). Y is converted to binary 0 or 1 and stored in Y. ob , our vector of observed targets. We then predict the targets from the test set, convert them again to binary 0 or 1 and store them in Y. pr . For comparing the observation with the prediction, we use the confusionMatrix function from the caret package.
A confusion matrix of a binary classifier is simply a 2×2 matrix that tells how many 0’s and how many 1’s had been predicted wrongly and correctly. A lot of metrics are derived from the matrix and printed in the lines above. The most important at the moment is the 62% prediction accuracy . This may hint that I bashed price action trading a little prematurely. But of course the 62% might have been just luck. We’ll see that later when we run a WFO test.
A final advice: R packages are occasionally updated, with the possible consequence that previous R code suddenly might work differently, or not at all. This really happens, so test carefully after any update.
Step 7: The strategy.
Now that we’ve tested our algorithm and got some prediction accuracy above 50% with a test data set, we can finally code our machine learning strategy. In fact we’ve already coded most of it, we just must add a few lines to the above Zorro script that exported the data set. This is the final script for training, testing, and (theoretically) trading the system ( DeepLearn. c ):
We’re using a WFO cycle of one year, split in a 90% training and a 10% out-of-sample test period. You might ask why I have earlier used two year’s data and a different split, 80/20, for calibrating the network in step 5. This is for using differently composed data for calibrating and for walk forward testing. If we used exactly the same data, the calibration might overfit it and compromise the test.
The selected WFO parameters mean that the system is trained with about 225 days data, followed by a 25 days test or trade period. Thus, in live trading the system would retrain every 25 days, using the prices from the previous 225 days. In the literature you’ll sometimes find the recommendation to retrain a machine learning system after any trade, or at least any day. But this does not make much sense to me. When you used almost 1 year’s data for training a system, it can obviously not deteriorate after a single day. Or if it did, and only produced positive test results with daily retraining, I would strongly suspect that the results are artifacts by some coding mistake.
Training a deep network takes really a long time, in our case about 10 minutes for a network with 3 hidden layers and 200 neurons. In live trading this would be done by a second Zorro process that is automatically started by the trading Zorro. In the backtest, the system trains at any WFO cycle. Therefore using multiple cores is recommended for training many cycles in parallel. The NumCores variable at -1 activates all CPU cores but one. Multiple cores are only available in Zorro S, so a complete walk forward test with all WFO cycles can take several hours with the free version.
In the script we now train both long and short trades. For this we have to allow hedging in Training mode, since long and short positions are open at the same time. Entering a position is now dependent on the return value from the advise function, which in turn calls either the neural. train or the neural. predict function from the R script. So we’re here entering positions when the neural net predicts a result above 0.5.
The R script is now controlled by the Zorro script (for this it must have the same name, NeuralLearn. r , only with different extension). It is identical to our R script above since we’re using the same network parameters. Only one additional function is needed for supporting a WFO test:
The neural. save function stores the Models list – it now contains 2 models for long and for short trades – after every training run in Zorro’s Data folder. Since the models are stored for later use, we do not need to train them again for repeated test runs.
This is the WFO equity curve generated with the script above (EUR/USD, without trading costs):
EUR/USD equity curve with 50-100-50 network structure.
Although not all WFO cycles get a positive result, it seems that there is some predictive effect. The curve is equivalent to an annual return of 89%, achieved with a 50-100-50 hidden layer structure. We’ll check in the next step how different network structures affect the result.
Since the neural. init , neural. train , neural. predict , and neural. save functions are automatically called by Zorro’s adviseLong/adviseShort functions, there are no R functions directly called in the Zorro script. Thus the script can remain unchanged when using a different machine learning method. Only the DeepLearn. r script must be modified and the neural net, for instance, replaced by a support vector machine. For trading such a machine learning system live on a VPS, make sure that R is also installed on the VPS, the needed R packages are installed, and the path to the R terminal set up in Zorro’s ini file. Otherwise you’ll get an error message when starting the strategy.
Step 8: The experiment.
If our goal had been developing a strategy, the next steps would be the reality check, risk and money management, and preparing for live trading just as described under model-based strategy development. But for our experiment we’ll now run a series of tests, with the number of neurons per layer increased from 10 to 100 in 3 steps, and 1, 2, or 3 hidden layers (deepnet does not support more than 3). So we’re looking into the following 9 network structures: c(10), c(10,10), c(10,10,10), c(30), c(30,30), c(30,30,30), c(100), c(100,100), c(100,100,100). For this experiment you need an afternoon even with a fast PC and in multiple core mode. Here are the results (SR = Sharpe ratio, R2 = slope linearity):
We see that a simple net with only 10 neurons in a single hidden layer won’t work well for short-term prediction. Network complexity clearly improves the performance, however only up to a certain point. A good result for our system is already achieved with 3 layers x 30 neurons. Even more neurons won’t help much and sometimes even produce a worse result. This is no real surprise, since for processing only 8 inputs, 300 neurons can likely not do a better job than 100.
Conclusão.
Our goal was determining if a few candles can have predictive power and how the results are affected by the complexity of the algorithm. The results seem to suggest that short-term price movements can indeed be predicted sometimes by analyzing the changes and ranges of the last 4 candles. The prediction is not very accurate – it’s in the 58%..60% range, and most systems of the test series become unprofitable when trading costs are included. Still, I have to reconsider my opinion about price action trading. The fact that the prediction improves with network complexity is an especially convincing argument for short-term price predictability.
It would be interesting to look into the long-term stability of predictive price patterns. For this we had to run another series of experiments and modify the training period ( WFOPeriod in the script above) and the 90% IS/OOS split. This takes longer time since we must use more historical data. I have done a few tests and found so far that a year seems to be indeed a good training period. The system deteriorates with periods longer than a few years. Predictive price patterns, at least of EUR/USD, have a limited lifetime.
Where can we go from here? There’s a plethora of possibilities, for instance:
Use inputs from more candles and process them with far bigger networks with thousands of neurons. Use oversampling for expanding the training data. Prediction always improves with more training samples. Compress time series f. i. with spectal analysis and analyze not the candles, but their frequency representation with machine learning methods. Use inputs from many candles – such as, 100 – and pre-process adjacent candles with one-dimensional convolutional network layers. Use recurrent networks. Especially LSTM could be very interesting for analyzing time series – and as to my knowledge, they have been rarely used for financial prediction so far. Use an ensemble of neural networks for prediction, such as Aronson’s “oracles” and “comitees”.
Papers / Articles.
(3) V. Perervenko, Selection of Variables for Machine Learning.
I’ve added the C and R scripts to the 2018 script repository. You need both in Zorro’s Strategy folder. Zorro version 1.474, and R version 3.2.5 (64 bit) was used for the experiment, but it should also work with other versions.
69 thoughts on “Better Strategies 5: A Short-Term Machine Learning System”
I’ve tested your strategy using 30min AAPL data but “sae. dnn. train” returns all NaN in training.
(It works just decreasing neurons to less than (5,10,5)… but accuracy is 49%)
Can you help me to understand why?
Desde já, obrigado.
If you have not changed any SAE parameters, look into the. csv data. It is then the only difference to the EUR/USD test. Maybe something is wrong with it.
Another fantastic article, jcl. Zorro is a remarkable environment for these experiments. Thanks for sharing your code and your approach – this really opens up an incredible number of possibilities to anyone willing to invest the time to learn how to use Zorro.
The problem with AAPL 30min data was related to the normalizing method I used (X-mean/SD).
The features range was not between -1:1 and I assume that sae. dnn need it to work…
Anyway performances are not comparable to yours 🙂
I have one question:
why do you use Zorro for creating the features in the csv file and then opening it in R?
why not create the file with all the features in R in a few lines and do the training on the file when you are already in R? instead of getting inside Zorro and then to R.
When you want R to create the features, you must still transmit the price data and the targets from Zorro to R. So you are not gaining much. Creating the features in Zorro results usually in shorter code and faster training. Features in R make only sense when you need some R package for calculating them.
Really helpful and interesting article! I would like to know if there are any English version of the book:
“Das Börsenhackerbuch: Finanziell unabhängig durch algorithmische Handelssysteme”
I am really interested on it,
Not yet, but an English version is planned.
Thanks JCL! Please let me now when the English version is ready, because I am really interested on it.
Works superbly (as always). Muito Obrigado. One small note, if you have the package “dlm” loaded in R, TestOOS will fail with error: “Error in TestOOS() : cannot change value of locked binding for ‘X'”. This is due to there being a function X in the dlm package, so the name is locked when the package is loaded. Easily fixed by either renaming occurrences of the variable X to something else, or temporarily detaching the dlm package with: detach(“package:dlm”, unload=TRUE)
Thanks for the info with the dlm package. I admit that ‘X’ is not a particular good name for a variable, but a function named ‘X’ in a distributed package is even a bit worse.
Results below were generated by revised version of DeepSignals. r – only change was use of LSTM net from the rnn package on CRAN. The authors of the package regard their LSTM implementation as “experimental” and do not feel it is as yet learning properly, so hopefully more improvement to come there. (Spent ages trying to accomplish the LSTM element using the mxnet package but gave up as couldn’t figure out the correct input format when using multiple training features.)
Will post results of full WFO when I have finished LSTM version of DeepLearn. r.
Confusion Matrix and Statistics.
95% CI : (0.5699, 0.5956)
No Information Rate : 0.5002.
P-Value [Acc > NIR] : <2e-16.
Mcnemar's Test P-Value : 0.2438.
Pos Pred Value : 0.5844.
Neg Pred Value : 0.5813.
Detection Rate : 0.2862.
Detection Prevalence : 0.4897.
Balanced Accuracy : 0.5828.
Results of WFO test below. Again, only change to original files was the use of LSTM in R, rather than DNN+SAE.
Walk-Forward Test DeepLearnLSTMV4 EUR/USD.
Simulated account AssetsFix.
Bar period 1 hour (avg 87 min)
Simulation period 15.05.2018-07.06.2018 (12486 bars)
Test period 04.05.2018-07.06.2018 (6649 bars)
Lookback period 100 bars (4 days)
WFO test cycles 11 x 604 bars (5 weeks)
Training cycles 12 x 5439 bars (46 weeks)
Monte Carlo cycles 200.
Assumed slippage 0.0 sec.
Spread 0.0 pips (roll 0.00/0.00)
Contracts per lot 1000.0.
Gross win/loss 3628$ / -3235$ (+5199p)
Average profit 360$/year, 30$/month, 1.38$/day.
Max drawdown -134$ 34% (MAE -134$ 34%)
Total down time 95% (TAE 95%)
Max down time 5 weeks from Aug 2018.
Max open margin 40$
Max open risk 35$
Trade volume 5710964$ (5212652$/year)
Transaction costs 0.00$ spr, 0.00$ slp, 0.00$ rol.
Capital required 262$
Number of trades 6787 (6195/year, 120/week, 25/day)
Percent winning 57.6%
Max win/loss 16$ / -14$
Avg trade profit 0.06$ 0.8p (+12.3p / -14.8p)
Avg trade slippage 0.00$ 0.0p (+0.0p / -0.0p)
Avg trade bars 1 (+1 / -2)
Max trade bars 3 (3 hours)
Time in market 177%
Max open trades 3.
Max loss streak 17 (uncorrelated 11)
Annual return 137%
Profit factor 1.12 (PRR 1.08)
Sharpe ratio 1.79.
Kelly criterion 2.34.
R2 coefficient 0.435.
Ulcer index 13.3%
Prediction error 152%
Confidence level AR DDMax Capital.
Portfolio analysis OptF ProF Win/Loss Wgt% Cycles.
EUR/USD .219 1.12 3907/2880 100.0 XX/\//\X///
EUR/USD:L .302 1.17 1830/1658 65.0 /\/\//\////
EUR/USD:S .145 1.08 2077/1222 35.0 \//\//\\///
Interessante! For a still experimental LSTM implementation that result looks not bad.
Sorry for being completely off topic but could you please point me to the best place where i can learn to code trend lines?? I’m a complete beginner, but from trading experience i see them as an important part of what i would like to build…
Robot Wealth has an algorithmic trading course for that – you can find details on his blog robotwealth/.
I think you misunderstand the meaning pretrening. See my articles https://mql5/ru/articles/1103.
I think there is more fully described this stage.
I don’t think I misunderstood pretraining, at least not more than everyone else, but thanks for the links!
You can paste your LTSM r code please ?
Could you help me answering some questions?
I have few question below:
1.I want to test Commission mode.
If I use interactive broker, I should set Commission = ? in normal case.
2.If I press the “trade” button, I see the log the script will use DeepLearn_EURUSD. ml.
So real trade it will use DeepLearn_EURUSD. ml to get the model to trade?
And use neural. predict function to trade?
3.If I use the slow computer to train the data ,
I should move DeepLearn_EURUSD. ml to the trade computer?
I test the real trade on my interactive brokers and press the result button.
Can I use Commission=0.60 to train the neural and get the real result?
Result button will show the message below:
Trade Trend EUR/USD.
Bar period 2 min (avg 2 min)
Trade period 02.11.2018-02.11.2018.
Spread 0.5 pips (roll -0.02/0.01)
Contracts per lot 1000.0.
Commission should be normally not set up in the script, but entered in the broker specific asset list. Otherwise you had to change the script every time when you want to test it with a different broker or account. IB has different lot sizes and commissions, so you need to add the command.
to the script when you want to test it for an IB account.
Yes, DeepLearn_EURUSD. ml is the model for live trading, and you need to copy it to the trade computer.
Do I write assetList(“AssetsIB. csv”) in the right place?
So below code’s result includes Commission ?
I test the result with Commission that seems pretty good.
Annual +93% +3177p.
BarPeriod = 60; // 1 hour.
WFOPeriod = 252*24; // 1 year.
NumCores = -1; // use all CPU cores but one.
Spread = RollLong = RollShort = Commission = Slippage = 0;
if(Train) Hedge = 2;
I run the DeepLearn. c in the IB paper trade.
The code “LifeTime = 3; // prediction horizon” seems to close the position that you open after 3 bars(3 hours).
But I can’t see it close the position on third bar close.
I see the logs below:
Closing prohibited – check NFA flag!
[EUR/USD::L4202] Can’t close 1@1.10995 at 09:10:51.
In my IB paper trade, it the default order size is 1k on EUR/USD.
How to change the order size in paper trade?
Muito obrigado.
IB is an NFA compliant broker. You can not close trades on NFA accounts. You must set the NFA flag for opening a reverse position instead. And you must enable trading costs, otherwise including the commission has no effect. I don’t think that you get a positive result with trading costs.
Those account issues are not related to machine learning, and are better asked on the Zorro forum. Or even better, read the Zorro manual where all this is explained. Just search for “NFA”.
I do some experiment to change the neural’s parameter with commission.
The code is below:
BarPeriod = 60; // 1 hour.
WFOPeriod = 252*24; // 1 year.
NumCores = -1; // use all CPU cores but one.
Spread = RollLong = RollShort = Slippage = 0;
if(Train) Hedge = 2;
I get the result with commission that Annual Return is about +23%.
But I don’t complete understand the zorro’s setting and zorro’s report.
Walk-Forward Test DeepLearn EUR/USD.
Simulated account AssetsIB. csv.
Bar period 1 hour (avg 86 min)
Simulation period 15.05.2018-09.09.2018 (14075 bars)
Test period 23.04.2018-09.09.2018 (8404 bars)
Lookback period 100 bars (4 days)
WFO test cycles 14 x 600 bars (5 weeks)
Training cycles 15 x 5401 bars (46 weeks)
Monte Carlo cycles 200.
Simulation mode Realistic (slippage 0.0 sec)
Spread 0.0 pips (roll 0.00/0.00)
Contracts per lot 20000.0.
Gross win/loss 24331$ / -22685$ (+914p)
Average profit 1190$/year, 99$/month, 4.58$/day.
Max drawdown -1871$ 114% (MAE -1912$ 116%)
Total down time 92% (TAE 41%)
Max down time 18 weeks from Dec 2018.
Max open margin 2483$
Max open risk 836$
Trade volume 26162350$ (18916130$/year)
Transaction costs 0.00$ spr, 0.00$ slp, 0.00$ rol, -1306$ com.
Capital required 5239$
Number of trades 1306 (945/year, 19/week, 4/day)
Percent winning 52.5%
Max win/loss 375$ / -535$
Avg trade profit 1.26$ 0.7p (+19.7p / -20.3p)
Avg trade slippage 0.00$ 0.0p (+0.0p / -0.0p)
Avg trade bars 2 (+2 / -3)
Max trade bars 3 (3 hours)
Time in market 46%
Max open trades 3.
Max loss streak 19 (uncorrelated 10)
Annual return 23%
Profit factor 1.07 (PRR 0.99)
Sharpe ratio 0.56.
Kelly criterion 1.39.
R2 coefficient 0.000.
Ulcer index 20.8%
Confidence level AR DDMax Capital.
10% 29% 1134$ 4153$
20% 27% 1320$ 4427$
30% 26% 1476$ 4656$
40% 24% 1649$ 4911$
50% 23% 1767$ 5085$
60% 22% 1914$ 5301$
70% 21% 2245$ 5789$
80% 19% 2535$ 6216$
90% 16% 3341$ 7403$
95% 15% 3690$ 7917$
100% 12% 4850$ 9625$
Portfolio analysis OptF ProF Win/Loss Wgt% Cycles.
EUR/USD .256 1.07 685/621 100.0 /X/XXXXXXXXXXX.
The manual is your friend:
Great read…I built this framework to use XGB to analyze live ETF price movements. Let me know what you think:
Hi, deep learning researcher and programmer here. 🙂
Great blog and great article, congratulations! I have some comments:
& # 8211; if you use ReLUs as activation functions, pretraining is not necessary.
& # 8211; AE is genarraly referred to as networks with same input and output, I would call the proposed network rather a MLP (multi-layer perceptron).
Do you think it is possible to use Python (like TensorFlow) or LUA (like Torch7) based deep learing libraries with Zorro?
I have also heard that ReLUs make a network so fast that you can brute force train it in some cases, with no pretraining. But I have not yet experimented with that. The described network is commonly called ‘SAE’ since it uses autoencoders, with indeed the same number of inputs and outputs, for the pre-training process. & # 8211; I am not familiar with Torch7, but you can theoretically use Tensorflow with Zorro with a DLL based interface. The network structure must still be defined in Python, but Zorro can use the network for training and prediction.
Would you do YouTube Tutorials to your work, this series of articles. And where can I subscribe this kinda of algorithmic trading tutorials. Thanks for your contribution.
I would do YouTube tutorials if someone payed me very well for them. Until then, you can subscribe this blog with the link on the right above.
Why not feed economic data from a calendar like forexfactory into the net as well? I suggested that several times before. This data is what makes me a profitable manual trader (rookie though), if there is any intelligence in these neuronal networks it should improve performance greatly. input must be name (non farm payrolls for example or some unique identifier) , time left to release, predicted value (like 3-5 days before) last value and revision. Some human institutional traders claim its possible to trade profitably without a chart from this data alone. Detecting static support and resistance areas (horizontal lines) should be superior to any simple candle patterns. It can be mathematically modeled, as the Support and Resistance indicator from Point Zero Trading proves. Unfortunately i dont have a clue how Arturo the programmer did it. I imagine an artificial intelligence actually “seeing” what the market is focussed on (like speculation on a better than expected NFP report based on other positive Data in the days before, driving the dollar up into the report). “seeing” significant support and resistance levels should allow for trading risk, making reasonable decisions on where to place SL and TP.
We also made the experience that well chosen external data, not derived from the price curve, can improve the prediction. There is even a trading system based on Trump’s twitter outpourings. I can’t comment on support and resistance since I know no successful systems that use them, and am not sure that they exist at all.
thank you very much for everything that you did so far.
I read the book (German here, too) and am working through your blog articles right now.
I already learnt a lot and still am learning more and more about the really important stuff (other than: Your mindset must be perfect and you need to have well-defined goals. I never was a fan of such things and finally I found someone that is on the same opinion and actually teaches people how to correctly do it).
So, thank you very much and thanks in advance for all upcoming articles that I will read and you will post.
As a thank you I was thinking about sending you a corrected version of your book (there are some typos and wrong articles here and there…). Would you be interested in that?
Again thank you for everything and please keep up the good work.
Obrigado! And I’m certainly interested in a list of all my mistakes.
Thank you for this interesting post. I ran it on my pc and obtained similar results as yours. Then I wanted to see if it could perform as well when commission and rollover and slippage were included during test. I used the same figures as the ones used in the workshops and included in the AssetFix. csv file. The modifications I did in your DeepLearn. c file are as follows:
Spread = RollLong = RollShort = Commission = Slippage = 0;
The results then were not as optimistic as without commission:
Walk-Forward Test DeepLearn_realistic EUR/USD.
Simulated account AssetsFix.
Bar period 1 hour (avg 86 min)
Simulation period 09.05.2018-27.01.2017 (16460 bars)
Test period 22.04.2018-27.01.2017 (10736 bars)
Lookback period 100 bars (4 days)
WFO test cycles 18 x 596 bars (5 weeks)
Training cycles 19 x 5367 bars (46 weeks)
Monte Carlo cycles 200.
Simulation mode Realistic (slippage 5.0 sec)
Spread 0.5 pips (roll -0.02/0.01)
Contracts per lot 1000.0.
Gross win/loss 5608$ / -6161$ (-6347p)
Average profit -312$/year, -26$/month, -1.20$/day.
Max drawdown -635$ -115% (MAE -636$ -115%)
Total down time 99% (TAE 99%)
Max down time 85 weeks from Jun 2018.
Max open margin 40$
Max open risk 41$
Trade volume 10202591$ (5760396$/year)
Transaction costs -462$ spr, 46$ slp, -0.16$ rol, -636$ com.
Capital required 867$
Number of trades 10606 (5989/year, 116/week, 24/day)
Percent winning 54.9%
Max win/loss 18$ / -26$
Avg trade profit -0.05$ -0.6p (+11.1p / -14.8p)
Avg trade slippage 0.00$ 0.0p (+1.5p / -1.7p)
Avg trade bars 1 (+1 / -2)
Max trade bars 3 (3 hours)
Time in market 188%
Max open trades 3.
Max loss streak 19 (uncorrelated 12)
Annual return -36%
Profit factor 0.91 (PRR 0.89)
Sharpe ratio -1.39.
Kelly criterion -5.39.
R2 coefficient 0.737.
Ulcer index 100.0%
Confidence level AR DDMax Capital.
Portfolio analysis OptF ProF Win/Loss Wgt% Cycles.
EUR/USD .000 0.91 5820/4786 100.0 XX/\XX\X\X/X/\\X\\
I am a very beginner with Zorro, maybe I did a mistake ? O que você acha ?
No, your results look absolutely ok. The predictive power of 4 candles is very weak. This is just an experiment for finding out if price action has any predictive power at all.
Although it apparently has, I have not yet seen a really profitable system with this method. From the machine learning systems that we’ve programmed so far, all that turned out profitable used data from a longer price history.
Thank you for the great article, it’s exactly what I needed in order to start experimenting with ML in Zorro.
I’ve noticed that the results are slightly different each time despite using the random seed. Here it doesn’t matter thanks to the large number of trades but for example with daily bars the performance metrics fluctuate much more. My question is: do you happen to know from where does the randomness come? Is it still the training process in R despite the seed?
It is indeed so. Deepnet apparently uses also an internal function, not only the R random function, for randomizing some initial value.
any idea about how to use machine learning like in this example with indicators? you could do as better strategy 6.
would be very interesting.
Is it grid search inside the neural. train function allowed? I get error when I try it.
Besides Andy, how did you end up definining the LSTM structure using rnn? Is it not clear for me after reading inside the package.
onde é o código completo? (ou onde é o repositório?)
You said” Use genetic optimization for determining the most important signals just by the most profitable results from the prediction process. Great for curve fitting” How about after using genetic optimization process for determining the most profitable signals , match and measure the most profitable signals with distance metrics/similarity analysis(mutual information, DTW, frechet distance algorithm etc…) then use the distance metrics/similarity analysis as function for neural network prediction? Does that make sense ?
Distance to what? To each other?
Yes find similar profitable signal-patterns in history and find distance between patterns/profitable signals then predict the behavior of the profitable signal in the future from past patterns.
Was wondering about this point you made in Step 5:
“Our target is the return of a trade with 3 bars life time.”
But in the code, doesn’t.
mean that we are actually predicting the SIGN of the return, rather than the return itself?
Sim. Only the binary win/loss result, but not the magnitude of the win or loss is used for the prediction.
“When you used almost 1 year’s data for training a system, it can obviously not deteriorate after a single day. Or if it did, and only produced positive test results with daily retraining, I would strongly suspect that the results are artifacts by some coding mistake.”
There is an additional trap to be aware of related to jcl’s comment above that applies to supervised machine learning techniques (where you train a model against actual outcomes). Assume you are trying to predict the return three bars ahead (as in the example above – LifeTime = 3;). In real time you obviously don’t have access to the outcomes for one, two and three bars ahead with which to retrain your model, but when using historical data you do. With frequently retrained models (especially if using relatively short blocks of training data) it is easy to train a model offline (and get impressive results) with data you will not have available for training in real time. Then reality kicks in. Therefore truncating your offline training set by N bars (where N is the number of bars ahead you are trying to predict) may well be advisable…
Amazing work, could you please share the WFO code as well. I was able to run the code till neural. save but unable to generate the WFO results.
Muito obrigado.
The code above does use WFO.
Dear jcl, in the text you mentioned that you could predict the current leg of zig-zag indicator, could you please elaborate on how to do that? what features and responses would you reccomend?
I would never claim that I could predict the current leg of zigzag indicator. But we have indeed coded a few systems that attempted that. For this, simply use not the current price movement, but the current zigzag slope as a training target. Which parameters you use for the features is completely up to you.
Bom trabalho. I was wondering if you ever tried using something like a net long-short ratio of the asset (I. e. the FXCM SSI index – real time live data) as a feature to improve prediction?
Not with the FXCM SSI index, since it is not available as historical data as far as I know. But similar data of other markets, such as order book content, COT report or the like, have been used as features to a machine learning system.
I see, thanks, and whats’s the experience on those? do they have any predictive power? if you know any materials on this, I would be very interested to read it. (fyi, the SSI index can be exported from FXCM Trading Station (daily data from 2003 for most currency pairs)
Thanks for the info with the SSI. Yes, additional market data can have predictive power, especially from the order book. But since we gathered this experience with contract work for clients, I’m not at liberty to disclose details. However we plan an own study with ML evaluation of additional data, and that might result in an article on this blog.
Thanks jcl, looking forward to it! there is a way to record SSI ratios in a CSV file from a LUA Strategy script (FXCM’s scripting language) for live evaluation. happy to give you some details if you decide to evaluate this. (drop me an email) MyFxbook also has a similar indicator, but no historical data on that one unfortunately.
Does random forest algorithm have any advantage over deep net or neural networks for classification problems in financial data? I make it more clear ; I use number of moving averages and oscillators slope colour change for trading decision(buy - sell-hold).Sometimes one oscillator colour change is lagging other is faster etc..There is no problem at picking tops and bottoms but It is quite challenging to know when to hold. Since random forest doesnt’ need normalization, do they have any advantage over deep net or neural networks for classification? Thanks.
This depends on the system and the features, so there is no general answer. In the systems we did so far, a random forest or single decision tree was sometimes indeed better than a standard neural network, but a deep network beats anything, especially since you need not care as much about feature preselection. We meanwhile do most ML systems with deep networks.
I see thank you. I have seen some new implementations of LSTM which sounds interesting. One is called phased LSTM another one is from Yarin Gaal. He is using Bayesian technique(gaussian process) as dropout cs. ox. ac. uk/people/yarin. gal/website/blog_2248.html.
I hooked up the news flow from forexfactory into this algo and predictive power has improved by 7%.
I downloaded forexfactory news history from 2018. Used a algo to convert that into a value of -1 to 1 for EUR. This value becomes another parameter into the neural training network. I think there is real value there …let me see if we can get the win ratio to 75% and then I thik we have a real winner on hands here. …..
The neural training somehow only yields results with EURUSD.
Anyone tried GBPUSD or EURJPY.
That’s also my experience. There are only a few asset types with which price pattern systems seem to really work, and that’s mainly EUR/USD and some cryptos. We also had pattern systems with GBP/USD und USD/JPY, but they work less well and need more complex algos. Most currencies don’t expose patterns at all.

Option trading machine learning

Puxe pedidos 0.
Participe do GitHub hoje.
O GitHub é o lar de mais de 20 milhões de desenvolvedores que trabalham juntos para hospedar e rever o código, gerenciar projetos e criar software juntos.
Clone com HTTPS.
Use o Git ou o check-out com o SVN usando o URL da web.
Machine Learning Options Trading.
&cópia de; 2018 GitHub, Inc. Termos Privacidade Segurança Status Ajuda.
Você não pode executar essa ação neste momento.
Você fez login com outra guia ou janela. Recarregue para atualizar sua sessão. Você se separou em outra guia ou janela. Recarregue para atualizar sua sessão.

The Financial Hacker.
A new view on algorithmic trading.
Deep Learning Systems for Bitcoin – Parte 1.
Since December, bitcoins can not only be traded at more or less dubious exchanges, but also as futures at the CME and CBOE. And already several trading systems popped up for bitcoin and other cryptocurrencies. None of them can claim big success, with one exception. There is a strategy that easily surpasses all other bitcoin systems and probably also all known historical trading systems. Its name: Buy and Hold . In the light of the extreme success of that particular bitcoin strategy, do we really need any other trading system for cryptos? Continue reading “Deep Learning Systems for Bitcoin – Part 1”
Algorithmic Options Trading 3.
In this article we’ll look into a real options trading strategy, like the strategies that we code for clients. This one however is based on a system from a trading book. As mentioned before, options trading books often contain systems that really work – which can not be said about day trading or forex trading books. The system that we’ll examine here is indeed able to produce profits. Even extreme profits, since it apparently never loses . But it is also obvious that its author has never backtested it. Continue reading “Algorithmic Options Trading 3”
Hacking a HFT system.
Compared with machine learning or signal processing algorithms of conventional trading strategies, High Frequency Trading systems can be surprisingly simple. They need not attempt to predict future prices. They know the future prices already. Or rather, they know the prices that lie in the future for other, slower market participants. Recently we got some contracts for simulating HFT systems in order to determine their potential profit and maximum latency. This article is about testing HFT systems the hacker’s way. Continue reading “Hacking a HFT system”
Negociação de opções algorítmicas 2.
Nesta segunda parte da série de negociação de Opções Algorítmicas, analisaremos mais de perto os retornos das opções. Especialmente em combinar diferentes tipos de opções para obtenção de curvas de lucro e risco personalizadas. Os comerciantes de opções conhecem combinações com nomes engraçados como "Iron Condor" e # 8221; ou & # 8220; Butterfly & # 8221 ;, mas você não está limitado a eles. Com alguns truques, você pode criar instrumentos financeiros artificiais de qualquer propriedade desejada & # 8211; por exemplo & # 8220; Opções binárias & # 8221; com fator de pagamento de mais de 100%. Continue lendo & # 8220; Algorithmic Options Trading 2 & # 8221;
Bye Yahoo, and thanks for all the fish.
Just a quick post in the light of a very recent event. Users of financial functions of R, MatLab, Python, or Zorro got a bad surprise in the last days. Scripts and programs based on historical price data suddenly didn’t work anymore. And our favorite free historical price data provider, Yahoo, now responds on any access to their API in this way:
Algorithmic Options Trading 1.
Despite the many interesting features of options, private traders rarely take advantage of them (of course I’m talking here of serious options, not binary options). Maybe options are unpopular due to their reputation of being complex . Or due to their lack of support by most trading software tools. Or due to the price tags of the few tools that support them and of the historical data that you need for algorithmic trading. Whatever – we recently did several programming contracts for options trading systems, and I was surprised that even simple systems seemed to produce relatively consistent profit . Especially selling options appears more lucrative than trading ‘conventional’ instruments. This article is the first one of a mini-series about earning money with algorithmic options trading. Continue reading “Algorithmic Options Trading 1”
Melhores estratégias 5: um sistema de aprendizado de máquina a curto prazo.
It’s time for the 5th and final part of the Build Better Strategies series. In part 3 we’ve discussed the development process of a model-based system, and consequently we’ll conclude the series with developing a data-mining system. The principles of data mining and machine learning have been the topic of part 4. For our short-term trading example we’ll use a deep learning algorithm , a stacked autoencoder, but it will work in the same way with many other machine learning algorithms. With today’s software tools, only about 20 lines of code are needed for a machine learning strategy. I’ll try to explain all steps in detail. Continue reading “Better Strategies 5: A Short-Term Machine Learning System”
Get Rich Slowly.
Most trading systems are of the get-rich-quick type. They exploit temporary market inefficiencies and aim for annual returns in the 100% area. They require regular supervision and adaption to market conditions, and still have a limited lifetime. Their expiration is often accompanied by large losses. But what if you’ve nevertheless collected some handsome gains, and now want to park them in a more safe haven? Put the money under the pillow? Take it into the bank? Give it to a hedge funds? Obviously, all that goes against an algo trader’s honor code. Here’s an alternative. Continue reading “Get Rich Slowly”
Opções binárias: Scam ou Opportunity?
Estamos recentemente recebendo mais e mais contratos para codificar estratégias de opções binárias. O que nos dá uma consciência um pouco ruim, uma vez que essas opções são amplamente entendidas como um esquema para separar os comerciantes ingênuos de seu dinheiro. E seus corretores não fazem realmente nenhuma boa impressão no primeiro aspecto. Alguns são regulamentados em Chipre sob um endereço falso, outros não são regulados. Eles espalham histórias fabricadas sobre grandes lucros com robôs ou EAs. Eles dizem manipular suas curvas de preço para impedir que você vença. E se você ainda faz, alguns se recusam a pagar e eventualmente desaparecem sem rastro (mas com seu dinheiro). Aquelas são as histórias que você ouve sobre corretores de opções binárias. As opções binárias são apenas fraudes? Ou eles oferecem uma oportunidade oculta que até mesmo seus corretores geralmente não estão conscientes? Continue lendo & # 8220; Opções binárias: Scam ou Opportunity? & # 8221;
Better Strategies 4: Machine Learning.
Deep Blue was the first computer that won a chess world championship. That was 1996, and it took 20 years until another program, AlphaGo , could defeat the best human Go player. Deep Blue was a model based system with hardwired chess rules. AlphaGo is a data-mining system, a deep neural network trained with thousands of Go games. Not improved hardware, but a breakthrough in software was essential for the step from beating top Chess players to beating top Go players.
In this 4th part of the mini-series we’ll look into the data mining approach for developing trading strategies. This method does not care about market mechanisms. It just scans price curves or other data sources for predictive patterns. Machine learning or “Artificial Intelligence” is not always involved in data-mining strategies. In fact the most popular – and surprisingly profitable – data mining method works without any fancy neural networks or support vector machines. Continue reading “Better Strategies 4: Machine Learning”
Construa melhores estratégias! Part 3: The Development Process.
This is the third part of the Build Better Strategies series. In the previous part we’ve discussed the 10 most-exploited market inefficiencies and gave some examples of their trading strategies. In this part we’ll analyze the general process of developing a model-based trading system. As almost anything, you can do trading strategies in (at least) two different ways: There’s the ideal way , and there’s the real way . We begin with the ideal development process , broken down to 10 steps. Continue reading “Build Better Strategies! Part 3: The Development Process”
Dear Brokers…
Whatever software we’re using for automated trading: We all need some broker connection for the algorithm to receive price quotes and place trades. Seemingly a simple task. And almost any broker supports it through a protocol such as FIX, through an automated platform such as MT4™, or through a specific broker API. But if you think you can quickly hook up your trading software to a broker API, you’re up for a bad surprise. Dear brokers – please read this post and try to make hacker’s and coder’s lifes a little easier! Continue reading “Dear Brokers…”
Construa melhores estratégias! Part 2: Model-Based Systems.
Trading systems come in two flavors: model-based and data-mining . This article deals with model based strategies. Even when the basic algorithms are not complex, properly developing them has its difficulties and pitfalls (otherwise anyone would be doing it). A significant market inefficiency gives a system only a relatively small edge . Any little mistake can turn a winning strategy into a losing one. And you will not necessarily notice this in the backtest. Continue reading “Build Better Strategies! Part 2: Model-Based Systems”
Better Tests with Oversampling.
The more data you use for testing or training your strategy, the less bias will affect the test result and the more accurate will be the training. The problem: price data is always in short supply. Even shorter when you must put aside some part for out-of-sample tests. Extending the test or training period far into the past is not always a solution. The markets of the 1990s or 1980s were very different from today, so their price data can cause misleading results.
In this article I’ll describe a simple method to produce more trades for testing, training, and optimizing from the same amount of price data. The method is tested with a price action system based on data mining price patterns. Continue reading “Better Tests with Oversampling”
Construa melhores estratégias!
Basta postagens de blog, documentos e livros abordam como otimizar e testar os sistemas de negociação. But there is little information about how to get to such a system in the first place. The described strategies often seem to have appeared out of thin air. Does a trading system require some sort of epiphany? Or is there a systematic approach to developing it?
This post is the first of a small series in which I’ll attempt a methodical way to build trading strategies. The first part deals with the two main methods of strategy development, with market hypotheses and with a Swiss Franc case study. Continue reading “Build Better Strategies!”
The Cold Blood Index.
You’ve developed a new trading system. All tests produced impressive results. So you started it live. And are down by $2000 after 2 months. Or you have a strategy that worked for 2 years, but revently went into a seemingly endless drawdown. Situations are all too familiar to any algo trader. E agora? Carry on in cold blood, or pull the brakes in panic?
Several reasons can cause a strategy to lose money right from the start. It can be already expired since the market inefficiency disappeared. Or the system is worthless and the test falsified by some bias that survived all reality checks. Or it’s a normal drawdown that you just have to sit out. In this article I propose an algorithm for deciding very early whether or not to abandon a system in such a situation. Continue reading “The Cold Blood Index”
I Hired a Contract Coder.
You’re a trader with serious ambitions to use algorithmic methods. You already have an idea to be converted to an algorithm. The problem: You do not know to read or write code . So you hire a contract coder. A guy who’s paid for delivering a script that you can drop in your MT4, Ninja, TradeStation, or Zorro platform. Congratulations, now you’re an algorithmic trader. Just start the script and wait for the money to roll in. – Isso realmente funciona? Answer: it depends. Continue reading “I Hired a Contract Coder”
Is “Scalping” Irrational?
Clients often ask for strategies that trade on very short time frames . Some are possibly inspired by “I just made $2000 in 5 minutes” stories on trader forums. Others have heard of High Frequency Trading : the higher the frequency, the better must be the trading! The Zorro developers had been pestered for years until they finally implemented tick histories and millisecond time frames. Totally useless features? Or has short term algo trading indeed some quantifiable advantages? An experiment for looking into that matter produced a surprising result . Continue reading “Is “Scalping” Irrational?”
Hacker’s Tools.
For performing our financial hacking experiments (and for earning the financial fruits of our labor) we need some software machinery for research, testing, training, and live trading financial algorithms. No existing software platform today is really up to all those tasks. So you have no choice but to put together your system from different software packages. Fortunately, two are normally sufficient. I’ll use Zorro and R for most articles on this blog, but will also occasionally look into other tools. Continue reading “Hacker’s Tools”
Boosting Strategies with MMI.
Agora vamos repetir o nosso experimento com as 900 estratégias de negociação de tendências, mas desta vez com trades filtradas pelo Market Meanness Index. In our first experiment we found many profitable strategies, some even with high profit factors, but none of them passed White’s Reality Check . Então, todos eles provavelmente falharam no comércio real, apesar dos seus ótimos resultados no backtest. Desta vez, esperamos que o MMI melhore a maioria dos sistemas ao filtrar negócios em situações de mercado que não sejam tendências. Continue reading “Boosting Strategies with MMI”
The Market Meanness Index.
This indicator can improve – sometimes even double – the profit expectancy of trend following systems. The Market Meanness Index tells whether the market is currently moving in or out of a “trending” regime. It can this way prevent losses by false signals of trend indicators. É um algoritmo puramente estatístico e não baseado em volatilidade, tendências ou ciclos da curva de preços. Continue reading “The Market Meanness Index”
Seventeen Trade Methods That I Don’t Really Understand.
When I started with technical trading, I felt like entering the medieval alchemist scene. A multitude of bizarre trade methods and hundreds of technical indicators and lucky candle patterns promised glimpses into the future, if only of financial assets. I wondered – if a single one of them would really work, why would you need all the rest? And how can you foretell tomorrow’s price by drawing circles, angles, bats or butterflies on a chart? Continue reading “Seventeen Trade Methods That I Don’t Really Understand”
White’s Reality Check.
This is the third part of the Trend Experiment article series. We now want to evaluate if the positive results from the 900 tested trend following strategies are for real, or just caused by Data Mining Bias . But what is Data Mining Bias, after all? And what is this ominous White’s Reality Check ? Continue reading “White’s Reality Check”
The Trend Experiment.
This is the second part of the trend experiment article series, involving 900 systems and 10 different “smoothing” or “low-lag” indicators for finding out if trend really exists and can be exploited by a simple algorithmic system . When you do such an experiment, you have normally some expectations about the outcome, such as: Continue reading “The Trend Experiment”
Trend Indicators.
The most common trade method is dubbed ‘ going with the trend ‘. While it’s not completely clear how one can go with the trend without knowing it beforehand, most traders believe that ‘trend’ exists and can be exploited. ‘Trend’ is supposed to manifest itself in price curves as a sort of momentum or inertia that continues a price movement once it started. This inertia effect does not appear in random walk curves. Continue reading “Trend Indicators”
Money and How to Get It.
Contrary to popular belief, money is no material good. It is created out of nothing by banks lending it. Therefore, for each newly created lot of money there’s the same amount of debt . You’re destroying the money by repaying your credits. Since this requires a higher sum due to interest and compound interest, and since money is also permanently withdrawn from circulation by hoarding, the entire money supply must constantly grow. It must never shrink. If it still does, as in the 1930 economic crisis, loan defaults, bank crashes and bankruptcies are the result. The monetary system is therefore a classic Ponzi scheme . Continue reading “Money and How to Get It”

Forex Mecânico.
Negociação no mercado FX usando estratégias mecânicas de negociação.
Machine Learning in Forex Trading: Why many academics are doing it all wrong.
Building machine learning strategiesВ that can obtain decent results under live market conditions has always been an important challenge in algorithmic trading. В Despite the great amount of interest and the incredible potential rewards, there are still no academic publications that are able to show good machine learning models that can successfully tackle the trading problem in the real market (to the best of my knowledge, post a comment if you have one and I’ll be more than happy to read it). Although many papers published do seem to show promising results, it is often the case that these papers fall into a variety of different statistical bias problems that make the real market success of their machine learning strategies highly improbable. On today’s post I am going to talk about the problems that I see in academic research related with machine learning in Forex and how I believe this research could be improved to yield much more useful information for both the academic and trading communities.
Most pitfalls in machine learning strategy design when doing Forex trading are inevitably inherited from the world of deterministic learning problems. When building a machine learning algorithm for something like face recognition or letter recognition there is a well defined problem that does not change, which is generally tackled by building a machine learning model on a subset of the data (a training set) and then testing if the model was able to correctly solve the problem by using the reminder of the data (a testing set). This is why you have some famous and well established data-sets that can be used to establish the quality of newly developed machine learning techniques. The key point here however, is that the problems initially tackled by machine learning were mostly deterministic and time independent.
When moving into trading, applying this same philosophy yields manyВ problems related with both the partially non-deterministic character of the market and its time dependence. The mere act of attempting to select training and testing sets introduces a significant amount of bias (a data selection bias) that creates a problem. If the selection is repeated to improve results in the testing set – which you must assume happens in at least some cases – then the problem also adds a great amount of data-mining bias. The whole issue of doing a single training/validation exercise also generates a problem pertaining to how this algorithm is to be applied when live trading. By definition the live trading will be different since the selection of training/testing sets needs to be reapplied to different data (as now the testing set is truly unknown data). The bias inherent in the initial in-sample/out-of-sample period selection and the lack of any tested rules for trading under unknown data makes such techniques to commonly fail in live trading. If an algorithm is trained with 2000-2018 data and was cross validated with 2018-2018 data there is no reason to believe that the same success will happen if trained in 2003-2018 data and then live traded from 2018 to 2017, the data sets are very different in nature.
Measuring algorithm success is also a very relevant problem here. Inevitably the machine learning algorithms used for trading should be measured in merit by their ability to generate positive returns but some literature measures the merit of new algorithmic techniques by attempting to benchmark their ability to getВ correct predictions. Correct predictions do not necessarily equal profitable trading as you can easily see when building binary classifiers. If you attempt to predict the next candle’s direction you can still make a loss if you are mostly right on small candles and wrong on larger candles. As a matter of fact most of this type of classifiers – most of those that don’t work – end up predicting directionality with an above 50% accuracy, yetВ not above the level needed to surpass commissions that would permit profitable binary options trading.
To build strategiesВ that are mostly rid of the above problems I have always advocated for a methodology in which the machine learning algorithm is retrained before the making of any training decision. By using a moving window for training and never making more than one decision without retraining the entire algorithm we can get rid of the selection bias that is inherent in choosing a single in-sample/out-of-sample set. In this manner the whole test is a series of training/validation exercises which end up ensuring that the machine learning algorithm works even under tremendously different training data sets. I also advocate for the measuring of actual backtesting performance to measure a machine learning algorithm’s merit and furthermore I would go as far as to say that no algorithm can be worth its salt without being proven under real out-of-sample conditions. Developing algorithms in this manner is much harder and I haven’t found a single academic paper that follows this type of approach (if I missed it feel free to post a link so that I can include a comment!).
This does not mean that this methodology is completely problem free however, it is still subject to the classical problems relevant to all strategy building exercises, including curve-fitting bias and data-mining bias. This is why it is also important to use a large amount of data (I use 25+ years to test systems, always retraining after each machine learning derived decision) and to perform adequate data-mining bias evaluation tests to determine the confidence with which we can say that the results do not come from random chance. My friend AlgoTraderJo – who also happens to be a member of my trading community – is currently growing a thread at ForexFactory following this same type of philosophy for machine learning development, as we work on some new machine learning algorithms for my trading community. You can refer to his thread or past posts on my blog for several examples of machine learning algorithms developed in this manner.
If you would like to learn more about our developments in machine learning and how you too can also develop your own machine learning strategies using the F4 frameworkВ please consider joining Asirikuy, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.
5 Responses to “Machine Learning in Forex Trading: Why many academics are doing it all wrong”
Great article, the problems you highlight are certainly valid for system robustness!
A question I have, is it normal for say an EA to do exceedingly well in a certain pair and do terrible in all others?
Or, should a robust EA do well in at least several pairs, without any change in settings!
Thanks, for your great thoughts.
That question is interesting ;o). I believe that the question is better phrased as “can a system that survives on only one pair generate returns when live traded?” the answer is yes (both from theory and from my own experience). Having returns on only one pair does not mean that the system is “bad” it simply means that it exploits a historical inefficiency that is only present on one instrument. Provided you take care of bias sources (such as data-mining bias and curve-fitting bias) there is no reason why this will not work.
Now, if you have a system that works across many symbols then data-mining bias will be exponentially lower for an equal system that only works on one symbol and curve-fitting bias will also be lower due to the use of more data. So I would say that it’s better, but definitely not required.
But remember, measure your statistical biases!
I am so glad that you said it does not have to make a profit across all pairs! Also curve fitting, how does one know the limit of tweaking allowed before it becomes fitted?
Finally, I did a very simple test using the bog standard Moving Average EA on MT4, to see which pairs would react most widely to MAs. I backtested 52 pairs to see how many ‘Moving Period’between (1-20) would make a profit, regardless of drawdown. I wondered what you make of the results!
* 5 year test period.
* Drawdown not measured.
* 52 Pairs tested.
* Settings tested (Periods 1-20).
1 23 pairs, made no profit on any bar settings 1-20.
2 6 pairs, could only make profit on 1 setting.
3 14 pairs only, returned a profit on 5, or more different settings.
4 5 pairs only, returned a profit on 10, or more different settings.
1 BTCUSD 19 Settings out of 20, made a profit.
You need to make a distinction between curve-fitting bias and data-mining bias (or at least these two different types of bias, however you may want to call them). Curve-fitting bias is a bias created by finding an inefficiency across a set of data, it answers the question: is my system finding something general or something specific to the data I am using?. Data-mining bias answers the question: is my system finding a real historical inefficiency or are the results just because of my mining process (meaning coming from random chance)?
By increasing parameter spaces and degrees of freedom you are increasing data-mining bias (you are more likely to find a system just by chance, instead of a system that trades a real historical inefficiency). You can measure data-mining bias by using a test like White’s reality check. Doing this type of test is fundamental to reliable strategy design.
Read more about this distinction between biases here:
Also read this paper on the subject:
Before dwelling into the complexities of trading system design and finding strategies for trading I strongly advice getting a solid formation in statistics (coursera statistics courses are an excellent free start). Statistics will give you the power to analyse your own results and methodically address questions like these ;o)
[…] Machine Learning in Forex Trading: Why many academics are doing it all wrong [Mechanical Forex] Building machine learning strategies that can obtain decent results under live market conditions has always been an important challenge in algorithmic trading. Despite the great amount of interest and the incredible potential rewards, there are still no academic publications that are able to show good machine learning models that can successfully tackle the trading problem in the real m […]