US and EU stock indices data ingestion currently broken

An update on data ingestion and chart generation Here is a real quick update on what can currently be observed on the main website: Our main data source for US and EU stock indices changed some internals, which is why the automated data ingestion no longer works. Hence, the automatically generated charts stopped updating in June. Unfortunately we noticed very late, and now the charts for the US and EU stock indices are outdated. The charts for indices from India and all crypto-related things are not affected. What are the next steps to fix the US and EU stock indices charts? We are going to look for another data source and adapt all things accordingly. We will let you know as soon as everything is fixed.    

Working on a website for providing market breadth data

I have started to make plans regarding a simple website which presents market breadth data for "old school" stock indices and some crypto indices.

Current situation

Currently, a set of tools, a time-series database and a visualization tool run frequently to generate the results I've been presenting through screenshots on this blog.

Those mechanics run on a little private server for my own analytics purposes. There is no way I can grant third parties access to my private server, so I have to screenshot the graphs and upload them in this blog.

I realize that this is not a very good way of sharing information with you. 

This is why I've been thinking about other ways to publish the results of my calculations for market breadth data.

Setting the scope

I'm doing a lot of thinking about what I actually want to present to my small audience:
  • Do I want to present raw data only, being visualized by simple graphs?
  • Do I want to present insights, e.g. by marking divergences or possible weaknesses (or some chart formations etc.)?
  • Do I want to present analytics and real insights, based on my own experience?

If I would have enough time for this little project here, I would love to share my very own insights on charts and market breadth data, maybe even providing insights if I would time a certain investment or perform a trade.

But on the other hand, I have no professional background in the finance market, so I don't want anyone to risk money based on my insights.

In addition, time is something I don't have, actually. 

This narrows down the scope of a website. While I have many ideas what I could do (also what could be achieved when building a small team), I should be realistic and focus on what I can actually do.


First thoughts on a little website

So lack of time and professional background in the finance industry, I will build a small website which presents market breadth data without any registration required.

This little website should either provide a good charting tool with the market breadth data being classic indicators (which means I have to put more effort in it) or, maybe as a version 0.1, the website should present pre-rendered graphics which are similar to the screenshots I've been posting during the last weeks.

The website should also offer some form of navigation so visitors could browse through the various stock or crypto indices.

I am not sure yet if the data will be delayed by one trading day or nearly real-time since one of my data providers only hands out delayed data (1-2 trading days).

There would be a way to provide all data in real-time, but this would mean I'd have to re-write some parts of the tools and, well, I would have to buy access to some finance API. Unfortunately, the free real-time sources all lack some important data.

The crazy alternatives

Instead of creating a little website, I could also do one of the following things, but they all seem a little bit crazy or have their limits in some ways:

  • Setup a twitter bot
    • I could post all data on Twitter by using a bot which posts pre-rendered graphics
    • Pros: No website needed, use existing infrastructure, use something most of the users already know and use
    • Contra: There would be a lot of posts every day and it might be hard to find all the data you need if you're looking for a specific index
  • Setup a bot for updating this blog
    • I could setup a simple bot for writing blog posts with data screenshots automatically
    • Pros: No website needed, use existing infrastructure
    • Contra: Image upload is not so easy when you want to do it automatically on Blogger, blog posts are not the ideal way of presenting such data
  • Offer e-mailing services
    • Simply allow users to register with their e-mail address and select which indices they are interested in; a bot could then mail out all the data frequently
    • Pros: No website needed, everyone only receives the data they're interested in
    • Contra: Also not a good way of presenting data, e-mail is not so reliable, could be considered as spam (we all love "newsletter"-alike e-mails, right?)
  •  Create a copy of my small virtual server and provide everyone direct access to my data dashboard
    • Simply create another instance of my database and dashboard and allow visitors to access my data dashboard directly
    • Pros: No real website needed, solution could be up within one day, everyone could browse the data freely (even with time frames of their liking)
    • Contra: Monthly server costs while there is no income (I run no ads), current solution is not the best one for presenting financial data


Your ideas?

 Do you have any other ideas or want to add something? Feel free to let me know.

I might give you another update if there is something to know (e.g. if I started to work on a website).



Popular posts from this blog

US and EU stock indices data ingestion currently broken

Crypto Market Barometer released on

Nikkei 225 - rally break or turnaround?